# mulesoft MuleSoft Inference Connector - Full Documentation

<!-- document id=096e824ffe167eae701a204281abe8a9f86f5522cbfd1835573d2d8226e532b4 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/configuring-image-generation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# Configuring Image Generation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Image\] Generate (only Base64)** operation.

## Configure the Image Generate (only Base64) Operation

The **\[Image\] Generate (only Base64)** operation creates images from text descriptions using
AI-powered image generation models.

Apply the **\[Image\] Generate (only Base64)** operation in various scenarios, such as for:

- **Content Generation**

  Generate images for blog posts, articles, social media, research, or design projects.

To configure the **\[Image\] Generate (only Base64)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the text description of the image you want to generate.

This is the XML for this operation:

```xml
<ms-inference:image-generation
  doc:id="7e62e70e-eff7-4080-bb20-3d162bb84c39"
  config-ref="OpenAIConfig"
  doc:name="Image generation">
    <ms-inference:prompt>
      <![CDATA[A photorealistic image of a modern office building with glass facades, surrounded by landscaped gardens]]>
    </ms-inference:prompt>
</ms-inference:image-generation>
```

### Output Configuration

This operation responds with a JSON payload containing the generated image data and metadata about
the generation process. This is an example response:

```json
{
  "payload": {
    "response": "<Base64 of image>"
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, for example:

```json
{
  "model": "Model name used",
  "promptUsed": "Revised prompt from LLM"
}
```

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=13e61317e269a9457df7b5975345d748cdb13eb68336776d39b2c4231c35c7c6 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/configuring-image-generation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# Configuring Image Generation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Image\] Generate (only Base64)** operation.

## Configure the Image Generate (only Base64) Operation

The **\[Image\] Generate (only Base64)** operation creates images from text descriptions using
AI-powered image generation models.

Apply the **\[Image\] Generate (only Base64)** operation in various scenarios, such as for:

- **Content Generation**

  Generate images for blog posts, articles, social media, research, or design projects.

To configure the **\[Image\] Generate (only Base64)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the text description of the image you want to generate.

This is the XML for this operation:

```xml
<ms-inference:image-generation
  doc:id="7e62e70e-eff7-4080-bb20-3d162bb84c39"
  config-ref="OpenAIConfig"
  doc:name="Image generation">
    <ms-inference:prompt>
      <![CDATA[A photorealistic image of a modern office building with glass facades, surrounded by landscaped gardens]]>
    </ms-inference:prompt>
</ms-inference:image-generation>
```

### Output Configuration

This operation responds with a JSON payload containing the generated image data and metadata about
the generation process. This is an example response:

```json
{
  "payload": {
    "response": "<Base64 of image>"
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, for example:

```json
{
  "model": "Model name used",
  "promptUsed": "Revised prompt from LLM"
}
```

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=96adc72d540bb5b49eff42c04485b10ff8d104b22941394b13afe5fb5b64c2f0 url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/configuring-image-generation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# Configuring Image Generation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Image\] Generate (only Base64)** operation.

## Configure the Image Generate (only Base64) Operation

The **\[Image\] Generate (only Base64)** operation creates images from text descriptions using
AI-powered image generation models.

Apply the **\[Image\] Generate (only Base64)** operation in various scenarios, such as for:

- **Content Generation**

  Generate images for blog posts, articles, social media, research, or design projects.

To configure the **\[Image\] Generate (only Base64)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the text description of the image you want to generate.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs.

This is the XML for this operation:

```xml
<ms-inference:image-generation
  doc:id="7e62e70e-eff7-4080-bb20-3d162bb84c39"
  config-ref="OpenAIConfig"
  doc:name="Image generation">
    <ms-inference:prompt>
      <![CDATA[A photorealistic image of a modern office building with glass facades, surrounded by landscaped gardens]]>
    </ms-inference:prompt>
</ms-inference:image-generation>
```

### Output Configuration

This operation responds with a JSON payload containing the generated image data and metadata about
the generation process. This is an example response:

```json
{
  "payload": {
    "response": "<Base64 of image>"
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, for example:

```json
{
  "model": "Model name used",
  "promptUsed": "Revised prompt from LLM"
}
```

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/latest/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=53f6ac160e29075a47290a92d8d9b785870c52bfaf0d1ec130c5a5773f770ca1 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/configuring-moderation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# Configuring Moderation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Toxicity\] Detection by Text** operation.

## Configure the Toxicity Detection by Text Operation

The **\[Toxicity\] Detection by Text** operation classifies and scores any harmful content by the
user or the LLM.

Apply the **\[Toxicity\] Detection by Text** operation in various scenarios, such as for:

- **Toxic Inputs Detection**

  Detect and block toxic input by the user to prevent sending it to the LLM.

- **Harmful Responses Detection**

  Filter out LLM responses that could be considered toxic or offensive by users.

To configure the **\[Toxicity\] Detection by Text** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Text**

      Text to check for harmful content.

This is the XML for this operation:

```xml
<ms-inference:toxicity-detection-text
  doc:name="Toxicity detection text"
  doc:id="b5770a5b-d3f9-47ba-acec-ab0bd41e4188"
  config-ref="OpenAIConfig">
    <ms-inference:text>
      <![CDATA[You are fat]]>
    </ms-inference:text>
</ms-inference:toxicity-detection-text>
```

### Output Configuration

This operation responds with a JSON payload containing the toxicity detection and rating. This is an
example response:

```json
{
  "payload": {
    "flagged": true,
    "categories": [
      {
        "illicit/violent": 0.0000025466403947055455,
        "self-harm/instructions": 0.00023480495744356635,
        "harassment": 0.9798945372458964,
        "violence/graphic": 0.000005920916517463734,
        "illicit": 0.000013552078562406772,
        "self-harm/intent": 0.0002233150331012493,
        "hate/threatening": 0.0000012029639084557005,
        "sexual/minors": 0.0000024300240743279605,
        "harassment/threatening": 0.0007499928075102617,
        "hate": 0.00720390551996062,
        "self-harm": 0.0004822186797755494,
        "sexual": 0.00012644219446392274,
        "violence": 0.0004960569708019355
      }
    ]
  }
}
```

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=43e944cce813f0b3a0333ac741aef7e4b636d966efe0fe5dc8c93dfc488b4f1c url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/configuring-moderation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# Configuring Moderation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Toxicity\] Detection by Text** operation.

## Configure the Toxicity Detection by Text Operation

The **\[Toxicity\] Detection by Text** operation classifies and scores any harmful content by the
user or the LLM.

Apply the **\[Toxicity\] Detection by Text** operation in various scenarios, such as for:

- **Toxic Inputs Detection**

  Detect and block toxic input by the user to prevent sending it to the LLM.

- **Harmful Responses Detection**

  Filter out LLM responses that could be considered toxic or offensive by users.

To configure the **\[Toxicity\] Detection by Text** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Text**

      Text to check for harmful content.

This is the XML for this operation:

```xml
<ms-inference:toxicity-detection-text
  doc:name="Toxicity detection text"
  doc:id="b5770a5b-d3f9-47ba-acec-ab0bd41e4188"
  config-ref="OpenAIConfig">
    <ms-inference:text>
      <![CDATA[You are fat]]>
    </ms-inference:text>
</ms-inference:toxicity-detection-text>
```

### Output Configuration

This operation responds with a JSON payload containing the toxicity detection and rating. This is an
example response:

```json
{
  "payload": {
    "flagged": true,
    "categories": [
      {
        "illicit/violent": 0.0000025466403947055455,
        "self-harm/instructions": 0.00023480495744356635,
        "harassment": 0.9798945372458964,
        "violence/graphic": 0.000005920916517463734,
        "illicit": 0.000013552078562406772,
        "self-harm/intent": 0.0002233150331012493,
        "hate/threatening": 0.0000012029639084557005,
        "sexual/minors": 0.0000024300240743279605,
        "harassment/threatening": 0.0007499928075102617,
        "hate": 0.00720390551996062,
        "self-harm": 0.0004822186797755494,
        "sexual": 0.00012644219446392274,
        "violence": 0.0004960569708019355
      }
    ]
  }
}
```

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=843659c0170c2e315cd857949dbdb1b36b11498a3dd98c0a23d01c3fb554a395 url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/configuring-moderation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# Configuring Moderation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Toxicity\] Detection by Text** operation.

## Configure the Toxicity Detection by Text Operation

The **\[Toxicity\] Detection by Text** operation classifies and scores any harmful content by the
user or the LLM.

Apply the **\[Toxicity\] Detection by Text** operation in various scenarios, such as for:

- **Toxic Inputs Detection**

  Detect and block toxic input by the user to prevent sending it to the LLM.

- **Harmful Responses Detection**

  Filter out LLM responses that could be considered toxic or offensive by users.

To configure the **\[Toxicity\] Detection by Text** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Text**

      Text to check for harmful content.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs.

This is the XML for this operation:

```xml
<ms-inference:toxicity-detection-text
  doc:name="Toxicity detection text"
  doc:id="b5770a5b-d3f9-47ba-acec-ab0bd41e4188"
  config-ref="OpenAIConfig">
    <ms-inference:text>
      <![CDATA[You are fat]]>
    </ms-inference:text>
</ms-inference:toxicity-detection-text>
```

### Output Configuration

This operation responds with a JSON payload containing the toxicity detection and rating. This is an
example response:

```json
{
  "payload": {
    "flagged": true,
    "categories": [
      {
        "illicit/violent": 0.0000025466403947055455,
        "self-harm/instructions": 0.00023480495744356635,
        "harassment": 0.9798945372458964,
        "violence/graphic": 0.000005920916517463734,
        "illicit": 0.000013552078562406772,
        "self-harm/intent": 0.0002233150331012493,
        "hate/threatening": 0.0000012029639084557005,
        "sexual/minors": 0.0000024300240743279605,
        "harassment/threatening": 0.0007499928075102617,
        "hate": 0.00720390551996062,
        "self-harm": 0.0004822186797755494,
        "sexual": 0.00012644219446392274,
        "violence": 0.0004960569708019355
      }
    ]
  }
}
```

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/latest/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=a90acd0d8b5ab3e6b4a562a2347043beb7d7965b242124f6a3ca72503336d433 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/configuring-text-generation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# Configuring Text Generation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Agent\] Define Prompt Template**, **\[Chat\] Answer Prompt**, **\[Chat\]
Completions**, and **\[Tools\] Native Template (Reasoning only)** operations.

## Configure the Agent Define Prompt Template Operation

The **\[Agent\] Define Prompt Template** operation is essential for configuring specific prompt
templates with the LLM of your choice. This operation allows you to define and compose AI functions
using plain text, enabling the creation of natural language prompts, generating responses,
extracting information, invoking other prompts, or performing any text-based task.

Apply the **\[Agent\] Define Prompt Template** operation in various scenarios, such as for:

- **Customer Service Agents**

  Enhance customer service by providing case summaries, case classifications, summarizing large
  datasets, and more.

- **Sales Operation Agents**

  Assist sales teams in writing sales emails, summarizing cases for specific accounts, assessing the
  probability of closing deals, and more.

- **Marketing Agents**

  Support marketing teams in generating product descriptions, creating newsletters, planning social
  media campaigns, and more.

To configure the **\[Agent\] Define Prompt Template** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Template**

      Enter the prompt template for the operation.

    - **Instructions**

      Provide the instructions for the inference provider, outlining the goals of the task.

    - **Data**

      Specify the data for the inference provider to evaluate using the provided template and
      instructions.

This is the XML for this operation:

```xml
<ms-inference:agent-define-prompt-template doc:name="Agent define prompt template" doc:id="5944353c-c784-4268-9f16-c036e5eaf8e3" config-ref="OpenAIConfig" >
			<ms-inference:template ><![CDATA[You are a customer satisfaction agent, who analyses the customer feedback in the dataset. Answer via json output and add a type for the result only with positive or negative as well as the complete answer]]></ms-inference:template>
			<ms-inference:instructions ><![CDATA[If the customer feedback in the dataset is negative, open a service satisfaction case and apologize to the customer. If the customer feedback in the dataset is positive, thank the customer and wish them a nice day. Do not repeat the feedback and be more direct starting the conversation with formal greetings]]></ms-inference:instructions>
			<ms-inference:data ><![CDATA[The training last week was amazing, we learned so much and the trainer was very friendly]]></ms-inference:data>
</ms-inference:agent-define-prompt-template>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "{\n  \"type\": \"positive\",\n  \"response\": \"Thank you for your positive feedback on the training last week. We are glad to hear that you had a great experience. Have a nice day!\"\n}"
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "tokenUsage": {
      "inputCount": 9
      "outputCount": 9,
      "totalCount": 18,
  },
  "additionalAttributes": {
    "finish_reason": "stop",
    "model": "openai/gpt-4o-mini",
    "id": "agentcmpl-fc4137f6-0b40-4018-936f-f0df6c0b5da1"
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

## Configure the Chat Answer Prompt Operation

The **\[Chat\] Answer Prompt** operation is a simple prompt request operation to the configured LLM.
It uses a plain text prompt as input and responds with a plain text answer.

Apply the **\[Chat\] Answer Prompt** operation in various scenarios, such as for:

- **Basic Chatbots**

  Answer simple user prompts.

- **Customer Service Queries**

  Provide direct answers to frequently asked questions.

To configure the **\[Chat\] Answer Prompt** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the prompt as plain text for the operation.

This is the XML for this operation:

```xml
<ms-inference:chat-answer-prompt doc:name="Chat answer prompt" doc:id="f513c329-d277-41e6-932c-f5032b3365ac" config-ref="OpenAIConfig" >
	<ms-inference:prompt ><![CDATA[What is the capital of Switzerland?]]></ms-inference:prompt>
</ms-inference:chat-answer-prompt>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "The capital of Switzerland is Bern. It's known for its well-preserved medieval old town, which is a UNESCO World Heritage site. Bern became the capital of Switzerland in 1848. The Swiss parliament, the Federal Assembly, is located in Bern."
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
    "attributes": {
        "tokenUsage": {
            "inputCount": 9,
            "outputCount": 9,
            "totalCount": 18
        },
        "additionalAttributes": {
            "finish_reason": "stop",
            "model": "gpt-4o-mini",
            "id": "chatanswer-gc2425f6-0b70-4018-936f-f0df6c0b5da1"
        }
    }
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

## Configure the Chat Completions Operation

The **\[Chat\] Completions** operation is useful when you want to retain conversation history for a
multi-user chat operation. MuleSoft Object Store or any database can hold previous user
conversation, which can be provided as messages into the **\[Chat\] Completions** operation.

Apply the **\[Chat\] Completions** operation in various scenarios, such as for:

- **Customer Support Chats**

  Retaining the context of ongoing support conversations.

- **Multi-user Chat Applications**

  Maintaining conversation history for different users.

- **Personal Assistants**

  Keeping track of user interactions to provide more relevant responses.

To configure the **\[Chat\] Completions** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Messages**

      Specify the conversation history as a JSON array.

This is the XML for this operation:

```xml
<set-variable value='#[%dw 2.0
	output application/json
	---
	[{
		"role": "assistant",
		"content": "You are a helpful assistant."
	},
	{
		"role": "user",
		"content": "What is the capital of Switzerland!"
	}
	]]' doc:name="Set Variable" doc:id="00231872-9564-4fad-b580-4f745c16ac9d" variableName="payload.data"/>
<ms-inference:chat-completions doc:name="Chat completions" doc:id="b2c68037-6af9-4e2a-9297-c57749a38193" config-ref="OpenAIConfig" >
	<ms-inference:messages ><![CDATA[#[payload.data]]]></ms-inference:messages>
</ms-inference:chat-completions>
```

> [!NOTE] You can declare the `payload.data` variable separately or you can inline it with the
> messages tag.

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "The capital of Switzerland is **Bern**. 🇨🇭 \n"
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage and additional metadata, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 22,
      "outputCount": 15,
      "totalCount": 37
    },
    "additionalAttributes": {
      "finish_reason": "stop",
      "model": "gpt-4o-mini",
      "id": "chatcmpl-fc2425f6-0b70-4018-936f-f0df6c0b5da1"
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

## Configure the Tools Native Template (Reasoning only) Operation

The **\[Tools\] Native Template (Reasoning only)** operation is useful if you want to create
autonomous agents that can use external tools whenever a prompt can’t be answered directly by the AI
model. This operation selects which tools should be used for execution.

Apply the **\[Tools\] Native Template (Reasoning only)** operation in various scenarios, such as
for:

- **Automating Routine Tasks**

  Create autonomous agents that handle routine tasks by calling appropriate APIs.

- **Customer Support**

  Automate responses to common queries by integrating tools that provide necessary information.

- **Inventory Management**

  Use tools to check inventory levels or order status based on user prompts.

- **Employee Management**

  Retrieve employee information or manage employee-related tasks through API calls.

- **Sales and Marketing**

  Access CRM data or manage leads and accounts efficiently using predefined tools.

To configure the **\[Tools\] Native Template (Reasoning only)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Template**

      Enter the template for the operation.

    - **Instructions**

      Enter the instructions for the operation.

    - **Data**

      Enter the prompt to send to the LLM, along the tools array. The result suggests a chaining of
      the tool as a request to fulfill the users request.

    - **Tools**

      Enter the tools for the operation.

This is an example of the tools array in the payload:

```json
{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_temperature",
        "description": "Get the current temperature for a specific location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g., San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["Celsius", "Fahrenheit"],
              "description": "The temperature unit to use. Infer this from the user's location."
            }
          },
          "required": ["location", "unit"]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_rain_probability",
        "description": "Get the probability of rain for a specific location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g., San Francisco, CA"
            }
          },
          "required": ["location"]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_delivery_date",
        "description": "Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {
              "type": "string",
              "description": "The customer's order ID."
            }
          },
          "required": ["order_id"]
        }
      }
    }
  ]
}
```

This is the XML for this operation:

```xml
<ms-inference:tools-native-template doc:name="Tools native template" doc:id="ff267b16-b0f7-4a8d-8dd8-004a8269862b" config-ref="OpenAIConfig">
    <ms-inference:template ><![CDATA[You are a helpful assistant that can use the following tools to answer questions.]]></ms-inference:template>
    <ms-inference:instructions ><![CDATA[Use the tools to answer the question.]]></ms-inference:instructions>
    <ms-inference:data ><![CDATA[#[payload.dataset]]]></ms-inference:data>
    <ms-inference:tools ><![CDATA[#[payload.tools]]]></ms-inference:tools>
</ms-inference:tools-native-template>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response.

#### Example Response Payload (Tools not used)

If the prompt is a general question that can be answered with public knowledge, such as
`What is the capital of Switzerland?`, the response is:

```json
{
  "response": "The capital of Switzerland is Bern."
}
```

#### Example Response Payload (Tools used)

If the prompt requires accessing external data through a tool, such as
`When will my order (ID = 220) be delivered?`, the AI model uses the configured tool to fetch the
information, and the response is:

```json
{
  "response": {
    "tools": [
      {
        "function": {
          "name": "get_delivery_date",
          "arguments": "{\"order_id\":\"220\"}"
        },
        "id": "call_ylw95cO0kxCt7NELy91cF7bX",
        "type": "function"
      }
    ]
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 204,
      "outputCount": 16,
      "totalCount": 220
    },
    "additionalAttributes": {
      "finish_reason": "tool_calls",
      "model": "gpt-4o-mini",
      "id": "chatcmpl-AQwXGmrzcO7XG5Ic1Hp0PKIrRoL4O"
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=4261829d2bcb6308c8d46940def0d4c9a88048e6676ead6900a8fa31a0e69731 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/configuring-text-generation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# Configuring Text Generation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Agent\] Define Prompt Template**, **\[Chat\] Answer Prompt**, **\[Chat\]
Completions**, and **\[Tools\] Native Template (Reasoning only)** operations.

## Configure the Agent Define Prompt Template Operation

The **\[Agent\] Define Prompt Template** operation is essential for configuring specific prompt
templates with the LLM of your choice. This operation allows you to define and compose AI functions
using plain text, enabling the creation of natural language prompts, generating responses,
extracting information, invoking other prompts, or performing any text-based task.

Apply the **\[Agent\] Define Prompt Template** operation in various scenarios, such as for:

- **Customer Service Agents**

  Enhance customer service by providing case summaries, case classifications, summarizing large
  datasets, and more.

- **Sales Operation Agents**

  Assist sales teams in writing sales emails, summarizing cases for specific accounts, assessing the
  probability of closing deals, and more.

- **Marketing Agents**

  Support marketing teams in generating product descriptions, creating newsletters, planning social
  media campaigns, and more.

To configure the **\[Agent\] Define Prompt Template** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Template**

      Enter the prompt template for the operation.

    - **Instructions**

      Provide the instructions for the inference provider, outlining the goals of the task.

    - **Data**

      Specify the data for the inference provider to evaluate using the provided template and
      instructions.

This is the XML for this operation:

```xml
<ms-inference:agent-define-prompt-template doc:name="Agent define prompt template" doc:id="5944353c-c784-4268-9f16-c036e5eaf8e3" config-ref="OpenAIConfig" >
			<ms-inference:template ><![CDATA[You are a customer satisfaction agent, who analyses the customer feedback in the dataset. Answer via json output and add a type for the result only with positive or negative as well as the complete answer]]></ms-inference:template>
			<ms-inference:instructions ><![CDATA[If the customer feedback in the dataset is negative, open a service satisfaction case and apologize to the customer. If the customer feedback in the dataset is positive, thank the customer and wish them a nice day. Do not repeat the feedback and be more direct starting the conversation with formal greetings]]></ms-inference:instructions>
			<ms-inference:data ><![CDATA[The training last week was amazing, we learned so much and the trainer was very friendly]]></ms-inference:data>
</ms-inference:agent-define-prompt-template>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "{\n  \"type\": \"positive\",\n  \"response\": \"Thank you for your positive feedback on the training last week. We are glad to hear that you had a great experience. Have a nice day!\"\n}"
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 12,
      "outputCount": 350,
      "totalCount": 362
    },
    "additionalAttributes": {
      "finishReason": "stop",
      "model": "openai/gpt-4o-mini",
      "id": "agentcmpl-fc4137f6-0b40-4018-936f-f0df6c0b5da1",
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## Configure the Chat Answer Prompt Operation

The **\[Chat\] Answer Prompt** operation is a simple prompt request operation to the configured LLM.
It uses a plain text prompt as input and responds with a plain text answer.

Apply the **\[Chat\] Answer Prompt** operation in various scenarios, such as for:

- **Basic Chatbots**

  Answer simple user prompts.

- **Customer Service Queries**

  Provide direct answers to frequently asked questions.

To configure the **\[Chat\] Answer Prompt** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the prompt as plain text for the operation.

This is the XML for this operation:

```xml
<ms-inference:chat-answer-prompt doc:name="Chat answer prompt" doc:id="f513c329-d277-41e6-932c-f5032b3365ac" config-ref="OpenAIConfig" >
	<ms-inference:prompt ><![CDATA[What is the capital of Switzerland?]]></ms-inference:prompt>
</ms-inference:chat-answer-prompt>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "The capital of Switzerland is Bern. It's known for its well-preserved medieval old town, which is a UNESCO World Heritage site. Bern became the capital of Switzerland in 1848. The Swiss parliament, the Federal Assembly, is located in Bern."
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 12,
      "outputCount": 350,
      "totalCount": 362
    },
    "additionalAttributes": {
      "finishReason": "stop",
      "model": "gpt-4o-mini",
      "id": "chatanswer-gc2425f6-0b70-4018-936f-f0df6c0b5da1",
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## Configure the Chat Completions Operation

The **\[Chat\] Completions** operation is useful when you want to retain conversation history for a
multi-user chat operation. MuleSoft Object Store or any database can hold previous user
conversation, which can be provided as messages into the **\[Chat\] Completions** operation.

Apply the **\[Chat\] Completions** operation in various scenarios, such as for:

- **Customer Support Chats**

  Retaining the context of ongoing support conversations.

- **Multi-user Chat Applications**

  Maintaining conversation history for different users.

- **Personal Assistants**

  Keeping track of user interactions to provide more relevant responses.

To configure the **\[Chat\] Completions** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Messages**

      Specify the conversation history as a JSON array. This field supports DataSense, which
      provides schema support.

This is the XML for this operation:

```xml
<set-variable value='#[%dw 2.0
	output application/json
	---
	[{
		"role": "assistant",
		"content": "You are a helpful assistant."
	},
	{
		"role": "user",
		"content": "What is the capital of Switzerland!"
	}
	]]' doc:name="Set Variable" doc:id="00231872-9564-4fad-b580-4f745c16ac9d" variableName="payload.data"/>
<ms-inference:chat-completions doc:name="Chat completions" doc:id="b2c68037-6af9-4e2a-9297-c57749a38193" config-ref="OpenAIConfig" >
	<ms-inference:messages ><![CDATA[#[payload.data]]]></ms-inference:messages>
</ms-inference:chat-completions>
```

> [!NOTE] You can declare the `payload.data` variable separately or you can inline it with the
> messages tag.

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "The capital of Switzerland is **Bern**. 🇨🇭 \n"
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage and additional metadata, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 12,
      "outputCount": 350,
      "totalCount": 362
    },
    "additionalAttributes": {
      "finishReason": "stop",
      "model": "gpt-4o-mini",
      "id": "chatcmpl-fc2425f6-0b70-4018-936f-f0df6c0b5da1"
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## Configure the Tools Native Template (Reasoning only) Operation

The **\[Tools\] Native Template (Reasoning only)** operation is useful if you want to create
autonomous agents that can use external tools whenever a prompt can’t be answered directly by the AI
model. This operation selects which tools should be used for execution.

Apply the **\[Tools\] Native Template (Reasoning only)** operation in various scenarios, such as
for:

- **Automating Routine Tasks**

  Create autonomous agents that handle routine tasks by calling appropriate APIs.

- **Customer Support**

  Automate responses to common queries by integrating tools that provide necessary information.

- **Inventory Management**

  Use tools to check inventory levels or order status based on user prompts.

- **Employee Management**

  Retrieve employee information or manage employee-related tasks through API calls.

- **Sales and Marketing**

  Access CRM data or manage leads and accounts efficiently using predefined tools.

To configure the **\[Tools\] Native Template (Reasoning only)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Template**

      Enter the template for the operation.

    - **Instructions**

      Enter the instructions for the operation.

    - **Data**

      Enter the prompt to send to the LLM, along the tools array. The result suggests a chaining of
      the tool as a request to fulfill the users request.

    - **Tools**

      Enter the tools for the operation. This field supports DataSense, which provides schema
      support.

This is an example of the tools array in the payload:

```json
{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_temperature",
        "description": "Get the current temperature for a specific location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g., San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["Celsius", "Fahrenheit"],
              "description": "The temperature unit to use. Infer this from the user's location."
            }
          },
          "required": ["location", "unit"]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_rain_probability",
        "description": "Get the probability of rain for a specific location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g., San Francisco, CA"
            }
          },
          "required": ["location"]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_delivery_date",
        "description": "Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {
              "type": "string",
              "description": "The customer's order ID."
            }
          },
          "required": ["order_id"]
        }
      }
    }
  ]
}
```

This is the XML for this operation:

```xml
<ms-inference:tools-native-template doc:name="Tools native template" doc:id="ff267b16-b0f7-4a8d-8dd8-004a8269862b" config-ref="OpenAIConfig">
    <ms-inference:template ><![CDATA[You are a helpful assistant that can use the following tools to answer questions.]]></ms-inference:template>
    <ms-inference:instructions ><![CDATA[Use the tools to answer the question.]]></ms-inference:instructions>
    <ms-inference:data ><![CDATA[#[payload.dataset]]]></ms-inference:data>
    <ms-inference:tools ><![CDATA[#[payload.tools]]]></ms-inference:tools>
</ms-inference:tools-native-template>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response.

#### Example Response Payload (Tools not used)

If the prompt is a general question that can be answered with public knowledge, such as
`What is the capital of Switzerland?`, the response is:

```json
{
  "response": "The capital of Switzerland is Bern."
}
```

#### Example Response Payload (Tools used)

If the prompt requires accessing external data through a tool, such as
`When will my order (ID = 220) be delivered?`, the AI model uses the configured tool to fetch the
information, and the response is:

```json
{
  "response": {
    "tools": [
      {
        "function": {
          "name": "get_delivery_date",
          "arguments": "{\"order_id\":\"220\"}"
        },
        "id": "call_ylw95cO0kxCt7NELy91cF7bX",
        "type": "function"
      }
    ]
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 204,
      "outputCount": 16,
      "totalCount": 220
    },
    "additionalAttributes": {
      "finishReason": "tool_calls",
      "model": "gpt-4o-mini",
      "id": "chatcmpl-AQwXGmrzcO7XG5Ic1Hp0PKIrRoL4O"
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=f4913beaa2d18a9a6f12871f79a13c6b46408ff76b452acdefab0ed034a974a1 url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/configuring-text-generation-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# Configuring Text Generation Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Agent\] Define Prompt Template**, **\[Chat\] Answer Prompt**, **\[Chat\]
Completions**, **\[Tools\] Native Template (Reasoning only)**, and **\[MCP\] Tooling** operations.

## Configure the Agent Define Prompt Template Operation

The **\[Agent\] Define Prompt Template** operation is essential for configuring specific prompt
templates with the LLM of your choice. This operation allows you to define and compose AI functions
using plain text, enabling the creation of natural language prompts, generating responses,
extracting information, invoking other prompts, or performing any text-based task.

Apply the **\[Agent\] Define Prompt Template** operation in various scenarios, such as for:

- **Customer Service Agents**

  Enhance customer service by providing case summaries, case classifications, summarizing large
  datasets, and more.

- **Sales Operation Agents**

  Assist sales teams in writing sales emails, summarizing cases for specific accounts, assessing the
  probability of closing deals, and more.

- **Marketing Agents**

  Support marketing teams in generating product descriptions, creating newsletters, planning social
  media campaigns, and more.

To configure the **\[Agent\] Define Prompt Template** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Template**

      Enter the prompt template for the operation.

    - **Instructions**

      Provide the instructions for the inference provider, outlining the goals of the task.

    - **Data**

      Specify the data for the inference provider to evaluate using the provided template and
      instructions.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs. The
    corresponding XML DSL tag for this operation is `<ms-inference:prompt-request-attributes>`.

This is the XML for this operation:

```xml
<ms-inference:agent-define-prompt-template doc:name="Agent define prompt template" doc:id="5944353c-c784-4268-9f16-c036e5eaf8e3" config-ref="OpenAIConfig" >
			<ms-inference:template ><![CDATA[You are a customer satisfaction agent, who analyses the customer feedback in the dataset. Answer via json output and add a type for the result only with positive or negative as well as the complete answer]]></ms-inference:template>
			<ms-inference:instructions ><![CDATA[If the customer feedback in the dataset is negative, open a service satisfaction case and apologize to the customer. If the customer feedback in the dataset is positive, thank the customer and wish them a nice day. Do not repeat the feedback and be more direct starting the conversation with formal greetings]]></ms-inference:instructions>
			<ms-inference:data ><![CDATA[The training last week was amazing, we learned so much and the trainer was very friendly]]></ms-inference:data>
</ms-inference:agent-define-prompt-template>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "{\n  \"type\": \"positive\",\n  \"response\": \"Thank you for your positive feedback on the training last week. We are glad to hear that you had a great experience. Have a nice day!\"\n}"
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 12,
      "outputCount": 350,
      "totalCount": 362
    },
    "additionalAttributes": {
      "finishReason": "stop",
      "model": "openai/gpt-4o-mini",
      "id": "agentcmpl-fc4137f6-0b40-4018-936f-f0df6c0b5da1",
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## Configure the Chat Answer Prompt Operation

The **\[Chat\] Answer Prompt** operation is a simple prompt request operation to the configured LLM.
It uses a plain text prompt as input and responds with a plain text answer.

Apply the **\[Chat\] Answer Prompt** operation in various scenarios, such as for:

- **Basic Chatbots**

  Answer simple user prompts.

- **Customer Service Queries**

  Provide direct answers to frequently asked questions.

To configure the **\[Chat\] Answer Prompt** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the prompt as plain text for the operation.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs. The
    corresponding XML DSL tag for this operation is `<ms-inference:chat-request-attributes>`.

This is the XML for this operation:

```xml
<ms-inference:chat-answer-prompt doc:name="Chat answer prompt" doc:id="f513c329-d277-41e6-932c-f5032b3365ac" config-ref="OpenAIConfig" >
	<ms-inference:prompt ><![CDATA[What is the capital of Switzerland?]]></ms-inference:prompt>
</ms-inference:chat-answer-prompt>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "The capital of Switzerland is Bern. It's known for its well-preserved medieval old town, which is a UNESCO World Heritage site. Bern became the capital of Switzerland in 1848. The Swiss parliament, the Federal Assembly, is located in Bern."
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 12,
      "outputCount": 350,
      "totalCount": 362
    },
    "additionalAttributes": {
      "finishReason": "stop",
      "model": "gpt-4o-mini",
      "id": "chatanswer-gc2425f6-0b70-4018-936f-f0df6c0b5da1",
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## Configure the Chat Completions Operation

The **\[Chat\] Completions** operation is useful when you want to retain conversation history for a
multi-user chat operation. MuleSoft Object Store or any database can hold previous user
conversation, which can be provided as messages into the **\[Chat\] Completions** operation.

Apply the **\[Chat\] Completions** operation in various scenarios, such as for:

- **Customer Support Chats**

  Retaining the context of ongoing support conversations.

- **Multi-user Chat Applications**

  Maintaining conversation history for different users.

- **Personal Assistants**

  Keeping track of user interactions to provide more relevant responses.

To configure the **\[Chat\] Completions** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Messages**

      Specify the conversation history as a JSON array. This field supports DataSense, which
      provides schema support.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs. The
    corresponding XML DSL tag for this operation is `<ms-inference:additional-request-attributes>`.

This is the XML for this operation:

```xml
<set-variable value='#[%dw 2.0
	output application/json
	---
	[{
		"role": "assistant",
		"content": "You are a helpful assistant."
	},
	{
		"role": "user",
		"content": "What is the capital of Switzerland!"
	}
	]]' doc:name="Set Variable" doc:id="00231872-9564-4fad-b580-4f745c16ac9d" variableName="payload.data"/>
<ms-inference:chat-completions doc:name="Chat completions" doc:id="b2c68037-6af9-4e2a-9297-c57749a38193" config-ref="OpenAIConfig" >
	<ms-inference:messages ><![CDATA[#[payload.data]]]></ms-inference:messages>
</ms-inference:chat-completions>
```

> [!NOTE] You can declare the `payload.data` variable separately or you can inline it with the
> messages tag.

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "response": "The capital of Switzerland is **Bern**. 🇨🇭 \n"
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage and additional metadata, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 12,
      "outputCount": 350,
      "totalCount": 362
    },
    "additionalAttributes": {
      "finishReason": "stop",
      "model": "gpt-4o-mini",
      "id": "chatcmpl-fc2425f6-0b70-4018-936f-f0df6c0b5da1"
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## Configure the Tools Native Template (Reasoning only) Operation

The **\[Tools\] Native Template (Reasoning only)** operation is useful if you want to create
autonomous agents that can use external tools whenever a prompt can’t be answered directly by the AI
model. This operation selects which tools should be used for execution.

Apply the **\[Tools\] Native Template (Reasoning only)** operation in various scenarios, such as
for:

- **Automating Routine Tasks**

  Create autonomous agents that handle routine tasks by calling appropriate APIs.

- **Customer Support**

  Automate responses to common queries by integrating tools that provide necessary information.

- **Inventory Management**

  Use tools to check inventory levels or order status based on user prompts.

- **Employee Management**

  Retrieve employee information or manage employee-related tasks through API calls.

- **Sales and Marketing**

  Access CRM data or manage leads and accounts efficiently using predefined tools.

To configure the **\[Tools\] Native Template (Reasoning only)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Template**

      Enter the template for the operation.

    - **Instructions**

      Enter the instructions for the operation.

    - **Data**

      Enter the prompt to send to the LLM, along the tools array. The result suggests a chaining of
      the tool as a request to fulfill the users request.

    - **Tools**

      Enter the tools for the operation. This field supports DataSense, which provides schema
      support.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs. The
    corresponding XML DSL tag for this operation is `<ms-inference:tools-request-attributes>`.

This is an example of the tools array in the payload:

```json
{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_temperature",
        "description": "Get the current temperature for a specific location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g., San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["Celsius", "Fahrenheit"],
              "description": "The temperature unit to use. Infer this from the user's location."
            }
          },
          "required": ["location", "unit"]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_rain_probability",
        "description": "Get the probability of rain for a specific location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g., San Francisco, CA"
            }
          },
          "required": ["location"]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_delivery_date",
        "description": "Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {
              "type": "string",
              "description": "The customer's order ID."
            }
          },
          "required": ["order_id"]
        }
      }
    }
  ]
}
```

This is the XML for this operation:

```xml
<ms-inference:tools-native-template doc:name="Tools native template" doc:id="ff267b16-b0f7-4a8d-8dd8-004a8269862b" config-ref="OpenAIConfig">
    <ms-inference:template ><![CDATA[You are a helpful assistant that can use the following tools to answer questions.]]></ms-inference:template>
    <ms-inference:instructions ><![CDATA[Use the tools to answer the question.]]></ms-inference:instructions>
    <ms-inference:data ><![CDATA[#[payload.dataset]]]></ms-inference:data>
    <ms-inference:tools ><![CDATA[#[payload.tools]]]></ms-inference:tools>
</ms-inference:tools-native-template>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response.

#### Example Response Payload (Tools not used)

If the prompt is a general question that can be answered with public knowledge, such as
`What is the capital of Switzerland?`, the response is:

```json
{
  "response": "The capital of Switzerland is Bern."
}
```

#### Example Response Payload (Tools used)

If the prompt requires accessing external data through a tool, such as
`When will my order (ID = 220) be delivered?`, the AI model uses the configured tool to fetch the
information, and the response is:

```json
{
  "response": {
    "tools": [
      {
        "function": {
          "name": "get_delivery_date",
          "arguments": "{\"order_id\":\"220\"}"
        },
        "id": "call_ylw95cO0kxCt7NELy91cF7bX",
        "type": "function"
      }
    ]
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 204,
      "outputCount": 16,
      "totalCount": 220
    },
    "additionalAttributes": {
      "finishReason": "tool_calls",
      "model": "gpt-4o-mini",
      "id": "chatcmpl-AQwXGmrzcO7XG5Ic1Hp0PKIrRoL4O"
      "promptFilterResults": [
        {
          "contentFilterResults": {
            "hate": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "jailbreak": {
              "severity": null,
              "filtered": false,
              "detected": false
            },
            "self_harm": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "sexual": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            },
            "violence": {
              "severity": "safe",
              "filtered": false,
              "detected": null
            }
          },
          "promptIndex": 0
        }
      ],
      "contentFilterResults": {
        "hate": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "protected_material_code": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "protected_material_text": {
          "severity": null,
          "filtered": false,
          "detected": false
        },
        "self_harm": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "sexual": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        },
        "violence": {
          "severity": "safe",
          "filtered": false,
          "detected": null
        }
      }
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

  - `promptFilterResults`: Flags if the user’s input contains potentially harmful content

  - `contentFilterResults`: Flags if the generated response contains potentially harmful content

## Configure the MCP Tooling Operation

The **\[MCP\] Tooling** operation allows you to execute tools hosted on an MCP server directly
through user-defined instructions. Instead of manually managing multiple connectors or workflows,
you can simply state what you want to achieve, and the LLM intelligently selects and runs the most
relevant tools. This selective execution model improves accuracy, reduces operational overhead, and
ensures users or agents interact only with the capabilities that matter — resulting in faster
execution, streamlined operations, and more predictable outcomes.

Apply the **\[MCP\] Tooling** operation in various scenarios, such as for:

- **IT Automation Agents**

  Automate routine IT tasks by executing MCP tools to trigger system diagnostics, restart services,
  fetch configuration data, and more.

- **Data Analysis Agents**

  Leverage MCP tools to extract metrics from databases, run statistical models, generate visual
  summaries, and more.

- **DevOps Agents**

  Streamline deployment workflows by invoking MCP tools to monitor CI/CD pipelines, retrieve logs,
  manage environment variables, and more.

- **Procurement Agents**

  Use MCP-based actions to check vendor pricing, generate purchase requests, validate contract
  statuses, and more.

- **SHR & Recruitment Agents**

  Automate candidate screening with MCP-enabled resume parsing, skill extraction, interview
  scheduling, and more.

As a prerequisite, ensure you download MCP Connector from Anypoint Exchange. Note that MuleSoft
Inference Connector now supports MCP Connector 1.2.

To configure the **\[MCP\] Tooling** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Mcp config references**

      Enter the list of MCP client configuration references.

    - **Mcp Client Config Ref**

      Enter the reference to the MCP client configuration defined as part of configuration from the
      MCP connector.

    - **Template**

      Enter the prompt template for the operation.

    - **Instructions**

      Provide the instructions for the inference provider, outlining the goals of the task.

    - **Data**

      Enter the prompt to send to the LLM, along the tools array. The result suggests a chaining of
      the tool as a request to fulfill the users request.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs. The
    corresponding XML DSL tag for this operation is `<ms-inference:mcp-request-attributes>`.

For more information about MCP Connector, see
[MCP Connector Reference](https://docs.mulesoft.com/mcp-connector/latest/mcp-connector-reference).

This is the XML for this operation:

```xml
<ms-inference:mcp-tools-native-template doc:name="[MCP] Tooling" doc:id="6783c672-bc72-49c7-bf3a-3f3379e3c9a9" config-ref="Text-generation-config"> <ms-inference:mcp-config-references> <ms-inference:mcp-config mcpClientConfigRef="MCP_Client_SSE_ERP" /> <ms-inference:mcp-config mcpClientConfigRef="MCP_Client_HTTP" /> </ms-inference:mcp-config-references> <ms-inference:template> <![CDATA[#['You are an helpful assistant']]]> </ms-inference:template> <ms-inference:instructions> <![CDATA[#['Answer the request with politeness']]]> </ms-inference:instructions> <ms-inference:data> <![CDATA[#[Check Inventory for MULETEST0]]]> </ms-inference:data> </ms-inference:mcp-tools-native-template>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{ "payload": {
"response": "The item MULETEST0 has 50 units in stock in the primary warehouse. Is there anything else I can help you with?",
"tool_used": "CheckInventory"
},
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
"attributes": {
    "tokenUsage": {
      "outputCount": 27,
      "totalCount": 1973,
      "inputCount": 1879
    },
    "additionalAttributes": {
      "promptFilterResults": null,
      "contentFilterResults": null,
      "finishReason": "STOP",
      "model": "gemini-2.5-flash",
      "id": "Em3aaPrJOceBmtkPvp3DuQo"
    }
  }
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

> [!NOTE] For the Gemini connection type, additional parameters must be included within the
> `GenerationConfig` property of the request payload.

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/latest/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=ada9b82939fadea03ac9c68623adefc8de764c3781ca12ead1b6910ec8575cf8 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/configuring-vision-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# Configuring Vision Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Image\] Read by (Url or Base64)** operation.

## Configure the Image Read by (Url or Base64) Operation

The **\[Image\] Read by (Url or Base64)** operation reads and interprets an image based on a prompt.

Apply the **\[Image\] Read by (Url or Base64)** operation in various scenarios, such as for:

- **Image Analysis**

  Analyze images in business reports, presentations, or customer service scenarios.

- **Content Generation**

  Describe images for blog posts, articles, or social media.

- **Visual Insights**

  Extract insights from images in research or design projects.

To configure the **\[Image\] Read by (Url or Base64)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the prompt for the operation.

    - **Image**

      Enter the URL or Base64 String of the image file that is to be read.

This is the XML for this operation:

```xml
<ms-inference:read-image
  doc:id="dfbd1a61-6e98-4b5b-b77a-bfe031e70d45"
  config-ref="OpenAIConfig"
  doc:name="Read image">
    <ms-inference:prompt>
      <![CDATA[Describe what you see in this image in detail]]>
    </ms-inference:prompt>
    <ms-inference:image-url>
      <![CDATA[https://example.com/image.png]]>
    </ms-inference:image-url>
</ms-inference:read-image>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "payload": {
    "response": "The image depicts the Eiffel Tower in Paris during a snowy day. The tower is partially covered in snow, and the surrounding trees and ground are also blanketed in snow. There is a pathway leading towards the Eiffel Tower, with a lamppost and some fencing along the sides. The overall scene has a serene and picturesque winter atmosphere."
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 267,
      "outputCount": 68,
      "totalCount": 335
    },
    "additionalAttributes": {
      "finish_reason": "stop",
      "model": "gpt-4o-mini",
      "id": "604ae573-8265-4dc0-b06e-457422f2fbd8"
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=8ca1b7decb299cffd11d488a54e708099cd9c8b48a370543a87e078cdbfd702d url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/configuring-vision-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# Configuring Vision Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Image\] Read by (Url or Base64)** operation.

## Configure the Image Read by (Url or Base64) Operation

The **\[Image\] Read by (Url or Base64)** operation reads and interprets an image based on a prompt.

Apply the **\[Image\] Read by (Url or Base64)** operation in various scenarios, such as for:

- **Image Analysis**

  Analyze images in business reports, presentations, or customer service scenarios.

- **Content Generation**

  Describe images for blog posts, articles, or social media.

- **Visual Insights**

  Extract insights from images in research or design projects.

To configure the **\[Image\] Read by (Url or Base64)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the prompt for the operation.

    - **Image**

      Enter the URL or Base64 String of the image file that is to be read.

This is the XML for this operation:

```xml
<ms-inference:read-image
  doc:id="dfbd1a61-6e98-4b5b-b77a-bfe031e70d45"
  config-ref="OpenAIConfig"
  doc:name="Read image">
    <ms-inference:prompt>
      <![CDATA[Describe what you see in this image in detail]]>
    </ms-inference:prompt>
    <ms-inference:image-url>
      <![CDATA[https://example.com/image.png]]>
    </ms-inference:image-url>
</ms-inference:read-image>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "payload": {
    "response": "The image depicts the Eiffel Tower in Paris during a snowy day. The tower is partially covered in snow, and the surrounding trees and ground are also blanketed in snow. There is a pathway leading towards the Eiffel Tower, with a lamppost and some fencing along the sides. The overall scene has a serene and picturesque winter atmosphere."
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 267,
      "outputCount": 68,
      "totalCount": 335
    },
    "additionalAttributes": {
      "finish_reason": "stop",
      "model": "gpt-4o-mini",
      "id": "604ae573-8265-4dc0-b06e-457422f2fbd8"
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=2d60ae347107b1fd7bbe710a0500efa2ec9dee3fb61e8caf0e366a1ce0792f0f url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/configuring-vision-operations.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# Configuring Vision Operations

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Configure the **\[Image\] Read by (Url or Base64)** operation.

## Configure the Image Read by (Url or Base64) Operation

The **\[Image\] Read by (Url or Base64)** operation reads and interprets an image based on a prompt.

Apply the **\[Image\] Read by (Url or Base64)** operation in various scenarios, such as for:

- **Image Analysis**

  Analyze images in business reports, presentations, or customer service scenarios.

- **Content Generation**

  Describe images for blog posts, articles, or social media.

- **Visual Insights**

  Extract insights from images in research or design projects.

To configure the **\[Image\] Read by (Url or Base64)** operation:

1.  Select the operation on the Anypoint Code Builder or Studio canvas.

2.  In the **General** properties tab for the operation, enter these values:

    - **Prompt**

      Enter the prompt for the operation.

    - **Image**

      Enter the URL or Base64 String of the image file that is to be read.

3.  In the **Additional Request Attributes** (optional) field, you can pass any additional request
    attributes in the request payload to generate more relevant and precise LLM outputs.

This is the XML for this operation:

```xml
<ms-inference:read-image
  doc:id="dfbd1a61-6e98-4b5b-b77a-bfe031e70d45"
  config-ref="OpenAIConfig"
  doc:name="Read image">
    <ms-inference:prompt>
      <![CDATA[Describe what you see in this image in detail]]>
    </ms-inference:prompt>
    <ms-inference:image-url>
      <![CDATA[https://example.com/image.png]]>
    </ms-inference:image-url>
</ms-inference:read-image>
```

### Output Configuration

This operation responds with a JSON payload containing the main LLM response. This is an example
response:

```json
{
  "payload": {
    "response": "The image depicts the Eiffel Tower in Paris during a snowy day. The tower is partially covered in snow, and the surrounding trees and ground are also blanketed in snow. There is a pathway leading towards the Eiffel Tower, with a lamppost and some fencing along the sides. The overall scene has a serene and picturesque winter atmosphere."
  }
}
```

The operation also returns attributes that aren’t within the main JSON payload, that include
information about token usage, for example:

```json
{
  "attributes": {
    "tokenUsage": {
      "inputCount": 267,
      "outputCount": 68,
      "totalCount": 335
    },
    "additionalAttributes": {
      "finish_reason": "stop",
      "model": "gpt-4o-mini",
      "id": "604ae573-8265-4dc0-b06e-457422f2fbd8"
    }
  }
}
```

- `tokenUsage`: Token usage metadata returned as attributes

  - `inputCount`: Number of tokens used to process the input

  - `outputCount`: Number of tokens used to generate the output

  - `totalCount`: Total number of tokens used for input and output

- `additionalAttributes`: Additional metadata from the LLM provider

  - `finish_reason`: The finish reason for the LLM response

  - `model`: The ID of the model used

  - `id`: The ID of the request

> [!NOTE] For Gemini Inference, additional parameters must be included within the `GenerationConfig`
> property of the request payload.

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/latest/)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=b7743bf354bfc7ac9e616f41570f3d08033be1b1f010e2e99af4d8f7f06ea2aa url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/mulesoft-inference-additional-config.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# MuleSoft Inference Connector - Additional Configuration Information

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

MuleSoft Inference Connector provides detailed configuration information for each operation type.
Select the operation you want to configure.

- [Configure Image Generation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/configuring-image-generation-operations)

- [Configure Moderation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/configuring-moderation-operations)

- [Configure Text Generation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/configuring-text-generation-operations)

- [Configure Vision Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/configuring-vision-operations)

<!-- document id=2ce51079f3e199ee77b44ce0c39e42f4099a4d2ab7320c18515275478414ea29 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/mulesoft-inference-additional-config.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# MuleSoft Inference Connector - Additional Configuration Information

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

MuleSoft Inference Connector provides detailed configuration information for each operation type.
Select the operation you want to configure.

- [Configure Image Generation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/configuring-image-generation-operations)

- [Configure Moderation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/configuring-moderation-operations)

- [Configure Text Generation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/configuring-text-generation-operations)

- [Configure Vision Operations](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/configuring-vision-operations)

<!-- document id=6e2805c43db1c9787566081917c089304a457ed3e488c3461ee1517fb43ef27d url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/mulesoft-inference-additional-config.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# MuleSoft Inference Connector - Additional Configuration Information

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

MuleSoft Inference Connector provides detailed configuration information for each operation type.
Select the operation you want to configure.

- [Configure Image Generation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/latest/configuring-image-generation-operations)

- [Configure Moderation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/latest/configuring-moderation-operations)

- [Configure Text Generation Operations](https://docs.mulesoft.com/mulesoft-inference-connector/latest/configuring-text-generation-operations)

- [Configure Vision Operations](https://docs.mulesoft.com/mulesoft-inference-connector/latest/configuring-vision-operations)

<!-- document id=9ed9fb5a332c0bd69c6d22cc3c1becf27eae5d4fc7ae85fa282e50a1e7f2fb0c url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/index.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# MuleSoft Inference Connector 1.0

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Anypoint Connector for MuleSoft Inference (MuleSoft Inference Connector) provides access to
inference offerings for large language models (LLMs) from multiple providers including OpenAI,
OpenAI Compatible Endpoints, OpenRouter, Heroku AI, Azure AI Foundry, Azure OpenAI, and many others.
This connector provides operations to interface directly with the API of various inference
providers, enabling seamless integration of AI capabilities into your Mule applications.

For information about compatibility and fixed issues, see the MuleSoft Inference Connector release
notes.

## Before You Begin

To use this connector, you must be familiar with:

- Anypoint Connectors

- Mule runtime engine (Mule)

- Elements and global elements in a Mule flow

- How to create a Mule app using Anypoint Code Builder or Anypoint Studio

Before creating an app, you must have:

- Java 17 (required for compilation and runtime)

- Apache Maven

- Credentials to access the MuleSoft Inference Connector target resource

- Anypoint Platform

- The latest versions of Anypoint Code Builder or Anypoint Studio

## Key Features

MuleSoft Inference Connector simplifies AI integration into Mule applications with:

- **Seamless Interaction with Inferenced LLMs**

  Integrate and utilize large language models (LLMs) effortlessly for tasks such as natural language
  processing, text generation, and more, including advanced features such as image analysis or
  generation, and toxicity detection in both user-provided and LLM-generated text.

- **Optimized Performance**

  Deliver high efficiency and performance in enterprise-grade Mule apps, ensuring smooth handling of
  AI operations.

- **Secure Integration**

  Connect corporate applications and data with innovative new AI offerings without damaging security
  posture. Provide full [TLS](https://docs.mulesoft.com/mule-runtime/latest/tls-configuration) support for secure Mule
  apps.

## Supported Inference Providers

- AI21Labs (beta)

- Anthropic (beta)

- Azure AI Foundry

- Azure OpenAI

- Cerebras (beta)

- Cohere (beta)

- Databricks (beta)

- DeepInfra (beta)

- DeepSeek (beta)

- Docker Models (beta)

- Fireworks (beta)

- GitHub Models (beta)

- GPT4ALL (beta)

- Groq AI (beta)

- Heroku AI

- Hugging Face (beta) (`hf-inference` only)

- LM Studio (beta)

- Mistral AI (beta)

- NVIDIA NIM (beta)

- Ollama (beta)

- OpenAI

- OpenAI Compatible Endpoints

- OpenRouter

- Perplexity (beta)

- Portkey (beta)

- Together.ai (beta)

- Vertex AI Express (beta)

- XAI (beta)

- Xinference (beta)

- ZHIPU AI (beta)

## Supported Moderation Providers

- Mistral AI (beta)

- OpenAI

## Supported Vision Model Providers

- Anthropic (beta)

- Azure AI Foundry

- GitHub Models (beta)

- Groq AI (beta)

- Hugging Face (beta) (`hf-inference` only)

- Mistral AI (beta)

- Ollama (beta)

- OpenAI

- OpenRouter

- Portkey (beta)

- Vertex AI Express (beta)

- XAI (beta)

## Supported Image Generation Providers

- Heroku AI

- Hugging Face (beta) (`hf-inference` only)

- OpenAI

- Stability AI (beta)

- XAI (beta)

> [!NOTE] To keep pace with the rapidly evolving AI landscape, certain LLMs are marked as beta.
> These are early-stage integrations that may change based on stability, demand, or provider
> updates. You can explore them but should do so with awareness that support might be limited and
> subject to change.

## Supported Operations by Offering

Not all operations are supported by each provider. This table provides a detailed view of operation
support across all providers.

<table><colgroup><col> <col> <col> <col> <col> <col> <col> <col></colgroup><thead><tr><th>LLM Provider</th><th>[Chat] Answer Prompt</th><th>[Chat] Completions</th><th>[Agent] Define Prompt Template</th><th>[Tools] Native Template (Reasoning only)*</th><th>[Image] Read by (Url or Base64)*</th><th>[Image] Generate (only Base64)*</th><th>[Toxicity] Detection by Text*</th></tr></thead><tbody><tr><td><p>AI21Labs</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Anthropic</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Azure AI Foundry</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Azure OpenAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Cerebras</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Cohere</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Databricks</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>DeepInfra</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>DeepSeek</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Docker Models</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Fireworks</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>GitHub Models</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>GPT4ALL</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Groq AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Heroku AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>Hugging Face</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>LM Studio</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Mistral AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>NVIDIA NIM</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Ollama</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>OpenAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td></tr><tr><td><p>OpenAI Compatible Endpoints</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>OpenRouter</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Perplexity</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Portkey</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Stability AI</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>Together.ai</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Vertex AI Express</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>XAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>Xinference</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Zhipu</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr></tbody></table>

> [!NOTE] `*` indicates that support depends on the specific model used.

## Next Step

After you complete the prerequisites, you are ready to create an app and configure the connector
using [Anypoint Studio](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-studio) or
[Anypoint Code Builder](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-config-acb).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Using Anypoint Code Builder to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-acb)

- [Using Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio)

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-additional-config)

- [MuleSoft Inference Connector Examples](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-examples)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=a3ff1ddb1d1b80380f79785186aa771a5074dbda579e7a0c36bcfcd19a19dd9a url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# MuleSoft Inference Connector 1.0 Reference

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

MuleSoft Inference Connector provides operations to interface directly with the API of various
inference providers, enabling seamless integration of AI capabilities into your Mule applications.

## Configurations

---

### Image Generation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#ImageGenerationConfig_HerokuImage">Heroku AI</a></p></li><li><p><a href="#ImageGenerationConfig_HuggingFaceImage">Hugging Face</a></p></li><li><p><a href="#ImageGenerationConfig_OpenaiImage">OpenAI</a></p></li><li><p><a href="#ImageGenerationConfig_StabilityAiImage">Stability AI</a></p></li><li><p><a href="#ImageGenerationConfig_XaiImage">xAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Heroku AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Heroku Inference Model</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Heroku Diffusion URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Heroku diffusion URL.</p></td><td><p><code>https://us.inference.heroku.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Stability AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Stability AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Image\] Generate (only Base64)](#GenerateImage)

---

### Moderation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#ModerationConfig_MistralaiModeration">Mistral AI</a></p></li><li><p><a href="#ModerationConfig_OpenaiModeration">OpenAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Toxicity\] Detection by Text](#ToxicityDetectionText)

---

### Text Generation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#TextGenerationConfig_Ai21labs">AI21Labs</a></p></li><li><p><a href="#TextGenerationConfig_Anthropic">Anthropic</a></p></li><li><p><a href="#TextGenerationConfig_AzureAiFoundry">Azure AI Foundry</a></p></li><li><p><a href="#TextGenerationConfig_AzureOpenai">Azure OpenAI</a></p></li><li><p><a href="#TextGenerationConfig_Cerebras">Cerebras</a></p></li><li><p><a href="#TextGenerationConfig_Cohere">Cohere</a></p></li><li><p><a href="#TextGenerationConfig_Databricks">Databricks</a></p></li><li><p><a href="#TextGenerationConfig_Deepinfra">DeepInfra</a></p></li><li><p><a href="#TextGenerationConfig_Deepseek">Deepseek</a></p></li><li><p><a href="#TextGenerationConfig_Docker">Docker</a></p></li><li><p><a href="#TextGenerationConfig_Fireworks">Fireworks</a></p></li><li><p><a href="#TextGenerationConfig_Github">GitHub</a></p></li><li><p><a href="#TextGenerationConfig_Gpt4all">GPT4All</a></p></li><li><p><a href="#TextGenerationConfig_Groq">Groq</a></p></li><li><p><a href="#TextGenerationConfig_Heroku">Heroku AI</a></p></li><li><p><a href="#TextGenerationConfig_HuggingFace">Hugging Face</a></p></li><li><p><a href="#TextGenerationConfig_Lmstudio">LM Studio</a></p></li><li><p><a href="#TextGenerationConfig_Mistralai">Mistral AI</a></p></li><li><p><a href="#TextGenerationConfig_Nvidia">Nvidia</a></p></li><li><p><a href="#TextGenerationConfig_Ollama">Ollama</a></p></li><li><p><a href="#TextGenerationConfig_Openai">OpenAI</a></p></li><li><p><a href="#TextGenerationConfig_OpenaiCompatible">OpenAI Compatible Endpoint</a></p></li><li><p><a href="#TextGenerationConfig_Openrouter">OpenRouter</a></p></li><li><p><a href="#TextGenerationConfig_Perplexity">Perplexity</a></p></li><li><p><a href="#TextGenerationConfig_Portkey">Portkey</a></p></li><li><p><a href="#TextGenerationConfig_Together">Together</a></p></li><li><p><a href="#TextGenerationConfig_VertexaiExpress">Vertex AI Express</a></p></li><li><p><a href="#TextGenerationConfig_Xai">xAI</a></p></li><li><p><a href="#TextGenerationConfig_Xinference">XInference</a></p></li><li><p><a href="#TextGenerationConfig_ZhipuAi">ZHIPU_AI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### AI21Labs

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ai21 Labs Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Anthropic

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Anthropic Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure AI Foundry

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure AI Foundry] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry resource name.</p></td><td></td><td></td></tr><tr><td><p>[Azure AI Foundry] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry API version.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure OpenAI] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure OpenAI resource name.</p></td><td></td><td></td></tr><tr><td><p>[Azure OpenAI] Deployment ID</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure OpenAI deployment ID.</p></td><td></td><td></td></tr><tr><td><p>[Azure OpenAI] User</p></td><td><div><div><p>String</p></div></div></td><td><p>Unique identifier representing your end-user, which can help to monitor and detect abuse.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Cerebras

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Cerebras Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Cohere

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Cohere Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Databricks

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>[Databricks] Serving Endpoint Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Databricks serving endpoint name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Databricks] Serving Endpoint URL host</p></td><td><div><div><p>String</p></div></div></td><td><p>Databricks serving endpoint URL host.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### DeepInfra

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Deep Infra Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Deepseek

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Deepseek Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Docker

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Docker Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Docker Models] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Docker models base URL.</p></td><td><p><code>http://localhost:12434</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Fireworks

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Fireworks Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GitHub

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Git Hub Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GPT4All

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Gpt4all Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[GPT4ALL] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>GPT4ALL base URL.</p></td><td><p><code>http://localhost:4891/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Groq

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Groq Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Heroku AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Heroku Inference Model</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td><p><code>claude-3-7-sonnet</code></p></td><td></td></tr><tr><td><p>Heroku Inference URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Inference URL.</p></td><td><p><code>https://us.inference.heroku.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### LM Studio

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Lm Studio Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[LM Studio] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>LM Studio base URL.</p></td><td><p><code>http://localhost:1234/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Nvidia

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Nvidia Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Ollama

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ollama Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Ollama] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Ollama base URL.</p></td><td><p><code>http://localhost:11434/api</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI model name.</p></td><td><p><code>gpt-4o-mini</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI Compatible Endpoint

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Compatible Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI compatible model name.</p></td><td><p><code>gpt-4o-mini</code></p></td><td></td></tr><tr><td><p>[OpenAI Compatible] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI compatible base URL.</p></td><td><p><code>https://server.endpoint.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenRouter

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open Router Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Perplexity

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Perplexity Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Portkey

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Portkey Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Portkey] Virtual Key</p></td><td><div><div><p>String</p></div></div></td><td><p>Portkey virtual key.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Together

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Together Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Vertex AI Express

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Vertex AI Express Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### XInference

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Inference Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Xinference] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Xinference base URL.</p></td><td><p><code>https://inference.top/api/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### ZHIPU\_AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Zhipu AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Agent\] Define Prompt Template](#AgentDefinePromptTemplate)

- [\[Chat\] Answer Prompt](#ChatAnswerPrompt)

- [\[Chat\] Completions](#ChatCompletions)

- [\[Tools\] Native Template (Reasoning only)](#ToolsNativeTemplate)

---

### Vision Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#VisionConfig_AnthropicVision">Anthropic</a></p></li><li><p><a href="#VisionConfig_AzureAiFoundryVision">Azure AI Foundry</a></p></li><li><p><a href="#VisionConfig_GithubVision">GitHub</a></p></li><li><p><a href="#VisionConfig_GroqVision">Groq</a></p></li><li><p><a href="#VisionConfig_HuggingFaceVision">Hugging Face</a></p></li><li><p><a href="#VisionConfig_MistralaiVision">Mistral AI</a></p></li><li><p><a href="#VisionConfig_OllamaVision">Ollama</a></p></li><li><p><a href="#VisionConfig_OpenaiVision">OpenAI</a></p></li><li><p><a href="#VisionConfig_OpenrouterVision">OpenRouter</a></p></li><li><p><a href="#VisionConfig_PortkeyVision">Portkey</a></p></li><li><p><a href="#VisionConfig_VertexaiExpressVision">Vertex AI Express</a></p></li><li><p><a href="#VisionConfig_XaiVision">xAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Anthropic

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Anthropic Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure AI Foundry

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure AI Foundry Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure AI Foundry] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry resource name.</p></td><td></td><td></td></tr><tr><td><p>[Azure AI Foundry] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry API version.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GitHub

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Git Hub Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Groq

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Groq Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Ollama

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ollama Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Ollama] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Ollama base URL.</p></td><td><p><code>http://localhost:11434/api</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenRouter

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open Router Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Portkey

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Portkey Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Portkey] Virtual Key</p></td><td><div><div><p>String</p></div></div></td><td><p>Portkey virtual key.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Vertex AI Express

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Vertex AI Express Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Image\] Read by (Url or Base64)](#ReadImage)

## Operations

## \[Image\] Generate (only Base64)

`<ms-inference:generate-image>`

This operation generates an image based on a prompt.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#ImageResponseAttributes">Image Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Image Generation Config](#ImageGenerationConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:IMAGE\_GENERATION\_FAILURE

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

## \[Toxicity\] Detection by Text

`<ms-inference:toxicity-detection-text>`

This operation detects harmful content in a text.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Text</p></td><td><div><div><p>Any</p></div></div></td><td><p>Text to moderate. Can be a single string or an array of strings.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr></tbody></table>

### For Configurations

- [Moderation Config](#ModerationConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOXICITY\_DETECTION\_OPERATION\_FAILURE

## \[Agent\] Define Prompt Template

`<ms-inference:agent-define-prompt-template>`

This operation defines a prompt template with instructions and data.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Template</p></td><td><div><div><p>String</p></div></div></td><td><p>Template string.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Instructions</p></td><td><div><div><p>String</p></div></div></td><td><p>Instructions for the LLM.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Data</p></td><td><div><div><p>String</p></div></div></td><td><p>Primary data content.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Chat\] Answer Prompt

`<ms-inference:chat-answer-prompt>`

This operation provides a simple chat answer for a single prompt.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Chat\] Completions

`<ms-inference:chat-completions>`

This operation provides chat completions by a messages array, including system or user messages
(conversation history).

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Messages</p></td><td><div><div><p>Any</p></div></div></td><td><p>Conversation history as a JSON array.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Tools\] Native Template (Reasoning only)

`<ms-inference:tools-native-template>`

This operation defines a tools template with instructions and data. This operation is useful if you
want to create autonomous agents that can use external tools whenever a prompt can’t be answered
directly by the AI model. This operation selects which tools can be used for execution.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Template</p></td><td><div><div><p>String</p></div></div></td><td><p>Template string.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Instructions</p></td><td><div><div><p>String</p></div></div></td><td><p>Instructions for the LLM.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Data</p></td><td><div><div><p>String</p></div></div></td><td><p>Primary data content.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Tools</p></td><td><div><div><p>Any</p></div></div></td><td><p>Tools configuration as a JSON array.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Image\] Read by (Url or Base64)

`<ms-inference:read-image>`

This operation reads an image from a URL or Base64 string.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Image</p></td><td><div><div><p>String</p></div></div></td><td><p>Image URL or Base64 string to send to the Vision Model.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Vision Config](#VisionConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:READ\_IMAGE\_OPERATION\_FAILURE

- MS-INFERENCE:RETRY\_EXHAUSTED

## Types

### TLS

Configures TLS to provide secure communications for the Mule app.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Enabled Protocols</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Comma-separated list of protocols enabled for this context.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Enabled Cipher Suites</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Comma-separated list of cipher suites enabled for this context.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Trust Store</strong></p></td><td><div><div><p><a href="#TrustStore">Truststore</a></p></div></div></td><td><div><div><p>Configures the TLS truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Key Store</strong></p></td><td><div><div><p><a href="#KeyStore">Keystore</a></p></div></div></td><td><div><div><p>Configures the TLS keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Revocation Check</strong></p></td><td><div><div><ul><li><p><a href="#StandardRevocationCheck">Standard Revocation Check</a></p></li><li><p><a href="#CustomOcspResponder">Custom OCSP Responder</a></p></li><li><p><a href="#CrlFile">CRL File</a></p></li></ul></div></div></td><td><div><div><p>Configures a revocation checking mechanism.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Truststore

Configures the truststore for TLS.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the truststore. Mule resolves the path relative to the current classpath and file system.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Type</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Type of truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Algorithm</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Encryption algorithm that the truststore uses.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Insecure</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>true</code>, Mule stops performing certificate validations. Setting this to <code>true</code> can make connections vulnerable to attacks.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Keystore

Configures the keystore for the TLS protocol. The keystore you generate contains a private key and a
public certificate.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the keystore. Mule resolves the path relative to the current classpath and file system.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Type</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Type of keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Alias</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Alias of the key to use when the keystore contains multiple private keys. By default, Mule uses the first key in the file.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Key Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the private key.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Algorithm</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Encryption algorithm that the keystore uses.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Standard Revocation Check

Configures standard revocation checks for TLS certificates.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Only End Entities</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Which elements to verify in the certificate chain:</p></div><div><ul><li><p><code>true</code></p><div><p>Verify only the last element in the certificate chain.</p></div></li><li><p><code>false</code></p><div><p>Verify all elements in the certificate chain.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prefer Crls</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>How to check certificate validity:</p></div><div><ul><li><p><code>true</code></p><div><p>Check the Certification Revocation List (CRL) for certificate validity.</p></div></li><li><p><code>false</code></p><div><p>Use the Online Certificate Status Protocol (OCSP) to check certificate validity.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>No Fallback</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Whether to use the secondary method to check certificate validity:</p></div><div><ul><li><p><code>true</code></p><div><p>Use the method that wasn’t specified in the <strong>Prefer Crls</strong> field (the secondary method) to check certificate validity.</p></div></li><li><p><code>false</code></p><div><p>Do not use the secondary method to check certificate validity.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Soft Fail</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>What to do if the revocation server can’t be reached or is busy:</p></div><div><ul><li><p><code>true</code></p><div><p>Avoid verification failure.</p></div></li><li><p><code>false</code></p><div><p>Allow the verification to fail.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Custom OCSP Responder

Configures a custom OCSP responder for certification revocation checks.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Url</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>URL of the OCSP responder.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Cert Alias</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Alias of the signing certificate for the OCSP response. If specified, the alias must be in the truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### CRL File

Specifies the location of the certification revocation list (CRL) file.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the CRL file.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Reconnection

Configures a reconnection strategy for an operation.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Fails Deployment</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Configures a reconnection strategy to use when a connector operation fails to connect to an external server.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Reconnection Strategy</strong></p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><div><div><p>Reconnection strategy to use.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Reconnect

Configures a standard reconnection strategy, which specifies how often to reconnect and how many
reconnection attempts the connector source or operation can make.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Frequency</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How often to attempt to reconnect, in milliseconds.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Blocking</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>false</code>, the reconnection strategy runs in a separate, non-blocking thread.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How many reconnection attempts the Mule app can make.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Reconnect Forever

Configures a forever reconnection strategy by which the connector source or operation attempts to
reconnect at a specified frequency for as long as the Mule app runs.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Frequency</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How often to attempt to reconnect, in milliseconds.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Blocking</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>false</code>, the reconnection strategy runs in a separate, non-blocking thread.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Expiration Policy

Configures an expiration policy strategy.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Max Idle Time</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Configures the maximum amount of time that a dynamic configuration instance can remain idle before Mule considers it eligible for expiration.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Time Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><div><div><p>Time unit for the <strong>Max Idle Time</strong> field.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Image Response Attributes

Configures image response attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Model</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Model.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prompt Used</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Prompt used.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Repeatable In Memory Stream

Configures the in-memory streaming strategy by which the request fails if the data exceeds the MAX
buffer size. Always run performance tests to find the optimal buffer size for your specific use
case.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Initial Buffer Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Initial amount of memory to allocate to the data stream. If the streamed data exceeds this value, the buffer expands by <strong>Buffer Size Increment</strong>, with an upper limit of <strong>Max Buffer Size</strong>.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Size Increment</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>This is by how much the buffer size expands if it exceeds its initial size. Setting a value of zero or lower means that the buffer should not expand, meaning that a <code>STREAM_MAXIMUM_SIZE_EXCEEDED</code> error is raised when the buffer gets full.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Max Buffer Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Maximum size of the buffer. If the buffer size exceeds this value, Mule raises a <code>STREAM_MAXIMUM_SIZE_EXCEEDED</code> error. A value of less than or equal to <code>0</code> means no limit.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>BYTE</p></li><li><p>KB</p></li><li><p>MB</p></li><li><p>GB</p></li></ul></div></div></td><td><div><div><p>Unit for the <strong>Initial Buffer Size</strong>, <strong>Buffer Size Increment</strong>, and <strong>Max Buffer Size</strong> fields.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Repeatable File Store Stream

Configures the repeatable file-store streaming strategy by which Mule keeps a portion of the stream
content in memory. If the stream content is larger than the configured buffer size, Mule backs up
the buffer’s content to disk and then clears the memory.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>In Memory Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Maximum amount of memory that the stream can use for data. If the amount of memory exceeds this value, Mule buffers the content to disk. To optimize performance:</p></div><div><ul><li><p>Configure a larger buffer size to avoid the number of times Mule needs to write the buffer on disk. This increases performance, but it also limits the number of concurrent requests your application can process, because it requires additional memory.</p></li><li><p>Configure a smaller buffer size to decrease memory load at the expense of response time.</p></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>BYTE</p></li><li><p>KB</p></li><li><p>MB</p></li><li><p>GB</p></li></ul></div></div></td><td><div><div><p>Unit for the <strong>In Memory Size</strong> field.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Error Mapping

Configures error mapping.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Source</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>ANY</p></li><li><p>REDELIVERY_EXHAUSTED</p></li><li><p>TRANSFORMATION</p></li><li><p>EXPRESSION</p></li><li><p>SECURITY</p></li><li><p>CLIENT_SECURITY</p></li><li><p>SERVER_SECURITY</p></li><li><p>ROUTING</p></li><li><p>CONNECTIVITY</p></li><li><p>RETRY_EXHAUSTED</p></li><li><p>TIMEOUT</p></li></ul></div></div></td><td><div><div><p>Source of the error.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Target</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Target of the error.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr></tbody></table>

### LLM Response Attributes

Configures LLM response attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Additional Attributes</strong></p></td><td><div><div><p><a href="#AdditionalAttributes">Additional Attributes</a></p></div></div></td><td><div><div><p>Additional attributes.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Token Usage</strong></p></td><td><div><div><p><a href="#TokenUsage">Token Usage</a></p></div></div></td><td><div><div><p>Token usage.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Additional Attributes

Configures additional attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Finish Reason</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Finish reason for the LLM response.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Id</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>ID of the request.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Model</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>ID of the model used.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Token Usage

Configures token usage metadata returned as attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Input Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Number of tokens used to process the input.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Output Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Number of tokens used to generate the output.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Total Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Total number of tokens used for input and output.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Proxy

Configures a proxy for outbound connections.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Host</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Host in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Port</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Port in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Username</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Username to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Non Proxy Hosts</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Non-proxy hosts.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### NTLM Proxy

Configures an NTLM proxy for outbound connections.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Ntlm Domain</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>NTLM domain.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Host</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Host in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Port</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Port in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Username</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Username to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Non Proxy Hosts</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Non-proxy hosts.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/)

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=cc96670f6bb35e831653421b91ae4c7e1ece104398f332d144763a6a5e9938eb url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/mulesoft-inference-connector-xml-maven.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# MuleSoft Inference Connector 1.0.0 XML and Maven Support

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Although you can manually code a Mule app in XML, it is more efficient to use Anypoint Code Builder
or Studio:

- If you manually code a Mule runtime engine (Mule) app in XML, either from the Anypoint Code
  Builder or Studio XML editor, or from a text editor, you can access the connector from your app by
  adding reference statements to both your XML Mule flow and the Apache Maven `pom.xml` file.

- If you add the connector in Anypoint Code Builder or Studio, the XML is automatically populated
  the code with the connector’s namespace and schema location and it also adds a dependency to the
  `pom.xml` file.

## Add a Namespace for the Connector

Paste the following code inside the `mule` tag of the header of your configuration XML:

```xml
http://www.mulesoft.org/schema/mule/mulesoft-inference
http://www.mulesoft.org/schema/mule/mulesoft-inference/current/mule-mulesoft-inference.xsd
```

This example shows how the namespace statements are placed in the `<mule>` XML block:

```xml
<mule xmlns:mulesoft-inference="http://www.mulesoft.org/schema/mule/mulesoft-inference"
	xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
	xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns:file="http://www.mulesoft.org/schema/mule/file"
	xmlns="http://www.mulesoft.org/schema/mule/core"
	xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core
	http://www.mulesoft.org/schema/mule/core/current/mule.xsd
	http://www.mulesoft.org/schema/mule/file
	http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
	http://www.mulesoft.org/schema/mule/http
	http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
	http://www.mulesoft.org/schema/mule/ee/core
	http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd
	http://www.mulesoft.org/schema/mule/mulesoft-inference
	http://www.mulesoft.org/schema/mule/mulesoft-inference/current/mule-mulesoft-inference.xsd">
```

## Add a POM File Dependency

The Apache Maven `pom.xml` file generated by Anypoint Studio contains dependencies for a Mule app.
If you code a Mule app manually, include this XML snippet in your `pom.xml` file to enable access to
this connector:

```xml
<dependency>
  <groupId>com.mulesoft.connectors</groupId>
  <artifactId>mule4-inference-connector</artifactId>
  <version>x.x.x</version>
  <classifier>mule-plugin</classifier>
</dependency>
```

Replace `x.x.x` with the version that corresponds to the connector you are using.

To obtain the most up-to-date `pom.xml` file information:

1.  Go to [Anypoint Exchange](https://www.mulesoft.com/exchange/).

2.  In Exchange, click **Login** and supply your Anypoint Platform username and password.

3.  In Exchange, search for `inference`.

4.  Select the connector.

5.  Click **Dependency Snippets** near the upper right of the screen.

## Next Step

After completing your namespace and pom.xml file, you can try the
[Examples](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-examples).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=a3fc3dc091c885b6ed358b781d289bab269b70f2d9f5849c4e60feb7ac3287bb url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/index.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# MuleSoft Inference Connector 1.1

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Anypoint Connector for MuleSoft Inference (MuleSoft Inference Connector) provides access to
inference offerings for large language models (LLMs) from multiple providers including OpenAI,
OpenAI Compatible Endpoints, OpenRouter, Heroku AI, Azure AI Foundry, Azure OpenAI, and many others.
This connector provides operations to interface directly with the API of various inference
providers, enabling seamless integration of AI capabilities into your Mule applications.

For information about compatibility and fixed issues, see the MuleSoft Inference Connector release
notes.

## Before You Begin

To use this connector, you must be familiar with:

- Anypoint Connectors

- Mule runtime engine (Mule)

- Elements and global elements in a Mule flow

- How to create a Mule app using Anypoint Code Builder or Anypoint Studio

Before creating an app, you must have:

- Java 17 (required for compilation and runtime)

- Apache Maven

- Credentials to access the MuleSoft Inference Connector target resource

- Anypoint Platform

- The latest versions of Anypoint Code Builder or Anypoint Studio

## Key Features

MuleSoft Inference Connector simplifies AI integration into Mule applications with:

- **Seamless Interaction with Inferenced LLMs**

  Integrate and utilize large language models (LLMs) effortlessly for tasks such as natural language
  processing, text generation, and more, including advanced features such as image analysis or
  generation, and toxicity detection in both user-provided and LLM-generated text.

- **Optimized Performance**

  Deliver high efficiency and performance in enterprise-grade Mule apps, ensuring smooth handling of
  AI operations.

- **Secure Integration**

  Connect corporate applications and data with innovative new AI offerings without damaging security
  posture. Provide full [TLS](https://docs.mulesoft.com/mule-runtime/latest/tls-configuration) support for secure Mule
  apps.

## Supported Inference Providers

- AI21Labs (beta)

- Anthropic (beta)

- Azure AI Foundry

- Azure OpenAI

- Cerebras (beta)

- Cohere (beta)

- Databricks (beta)

- DeepInfra (beta)

- DeepSeek (beta)

- Docker Models (beta)

- Fireworks (beta)

- Gemini (beta)

- GitHub Models (beta)

- GPT4ALL (beta)

- Groq AI (beta)

- Heroku AI

- Hugging Face (beta) (`hf-inference` only)

- LM Studio (beta)

- Mistral AI (beta)

- NVIDIA NIM (beta)

- Ollama (beta)

- OpenAI

- OpenAI Compatible Endpoints

- OpenRouter

- Perplexity (beta)

- Portkey (beta)

- Together.ai (beta)

- Vertex AI Express (beta)

- XAI (beta)

- Xinference (beta)

- ZHIPU AI (beta)

## Supported Moderation Providers

- Mistral AI (beta)

- OpenAI

## Supported Vision Model Providers

- Anthropic (beta)

- Azure AI Foundry

- Gemini (beta) (`Base64` only)

- GitHub Models (beta)

- Groq AI (beta)

- Hugging Face (beta) (`hf-inference` only)

- Mistral AI (beta)

- Ollama (beta)

- OpenAI

- OpenRouter

- Portkey (beta)

- Vertex AI Express (beta)

- XAI (beta)

## Supported Image Generation Providers

- Heroku AI

- Hugging Face (beta) (`hf-inference` only)

- OpenAI

- Stability AI (beta)

- XAI (beta)

> [!NOTE] To keep pace with the rapidly evolving AI landscape, certain LLMs are marked as beta.
> These are early-stage integrations that may change based on stability, demand, or provider
> updates. You can explore them but should do so with awareness that support might be limited and
> subject to change.

## Supported Operations by Offering

Not all operations are supported by each provider. This table provides a detailed view of operation
support across all providers.

<table><colgroup><col> <col> <col> <col> <col> <col> <col> <col></colgroup><thead><tr><th>LLM Provider</th><th>[Chat] Answer Prompt</th><th>[Chat] Completions</th><th>[Agent] Define Prompt Template</th><th>[Tools] Native Template (Reasoning only)*</th><th>[Image] Read by (Url or Base64)*</th><th>[Image] Generate (only Base64)*</th><th>[Toxicity] Detection by Text*</th></tr></thead><tbody><tr><td><p>AI21Labs</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Anthropic</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Azure AI Foundry</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Azure OpenAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Cerebras</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Cohere</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Databricks</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>DeepInfra</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>DeepSeek</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Docker Models</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Fireworks</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Gemini</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Base64 only</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>GitHub Models</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>GPT4ALL</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Groq AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Heroku AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>Hugging Face</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>LM Studio</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Mistral AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>NVIDIA NIM</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Ollama</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>OpenAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td></tr><tr><td><p>OpenAI Compatible Endpoints</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>OpenRouter</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Perplexity</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Portkey</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Stability AI</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>Together.ai</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Vertex AI Express</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>XAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td></tr><tr><td><p>Xinference</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Zhipu</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr></tbody></table>

> [!NOTE] `*` indicates that support depends on the specific model used.

## Next Step

After you complete the prerequisites, you are ready to create an app and configure the connector
using [Anypoint Studio](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-studio) or
[Anypoint Code Builder](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-config-acb).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Using Anypoint Code Builder to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-acb)

- [Using Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio)

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-additional-config)

- [MuleSoft Inference Connector Examples](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-examples)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=1725539d7aa967a2170d3226c9b091aea89321a27880bda780ff39d539aebcea url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# MuleSoft Inference Connector 1.1 Reference

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

MuleSoft Inference Connector provides operations to interface directly with the API of various
inference providers, enabling seamless integration of AI capabilities into your Mule applications.

## Configurations

---

### Image Generation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#ImageGenerationConfig_HerokuImage">Heroku AI</a></p></li><li><p><a href="#ImageGenerationConfig_HuggingFaceImage">Hugging Face</a></p></li><li><p><a href="#ImageGenerationConfig_OpenaiImage">OpenAI</a></p></li><li><p><a href="#ImageGenerationConfig_StabilityAiImage">Stability AI</a></p></li><li><p><a href="#ImageGenerationConfig_XaiImage">xAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Heroku AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Heroku Inference Model</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Heroku Diffusion URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Heroku diffusion URL.</p></td><td><p><code>https://us.inference.heroku.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Stability AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Stability AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Image\] Generate (only Base64)](#GenerateImage)

---

### Moderation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#ModerationConfig_MistralaiModeration">Mistral AI</a></p></li><li><p><a href="#ModerationConfig_OpenaiModeration">OpenAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Toxicity\] Detection by Text](#ToxicityDetectionText)

---

### Text Generation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#TextGenerationConfig_Ai21labs">AI21Labs</a></p></li><li><p><a href="#TextGenerationConfig_Anthropic">Anthropic</a></p></li><li><p><a href="#TextGenerationConfig_AzureAiFoundry">Azure AI Foundry</a></p></li><li><p><a href="#TextGenerationConfig_AzureOpenai">Azure OpenAI</a></p></li><li><p><a href="#TextGenerationConfig_Cerebras">Cerebras</a></p></li><li><p><a href="#TextGenerationConfig_Cohere">Cohere</a></p></li><li><p><a href="#TextGenerationConfig_Databricks">Databricks</a></p></li><li><p><a href="#TextGenerationConfig_Deepinfra">DeepInfra</a></p></li><li><p><a href="#TextGenerationConfig_Deepseek">Deepseek</a></p></li><li><p><a href="#TextGenerationConfig_Docker">Docker</a></p></li><li><p><a href="#TextGenerationConfig_Fireworks">Fireworks</a></p></li><li><p><a href="#TextGenerationConfig_Gemini">Gemini</a></p></li><li><p><a href="#TextGenerationConfig_Github">GitHub</a></p></li><li><p><a href="#TextGenerationConfig_Gpt4all">GPT4All</a></p></li><li><p><a href="#TextGenerationConfig_Groq">Groq</a></p></li><li><p><a href="#TextGenerationConfig_Heroku">Heroku AI</a></p></li><li><p><a href="#TextGenerationConfig_HuggingFace">Hugging Face</a></p></li><li><p><a href="#TextGenerationConfig_Lmstudio">LM Studio</a></p></li><li><p><a href="#TextGenerationConfig_Mistralai">Mistral AI</a></p></li><li><p><a href="#TextGenerationConfig_Nvidia">Nvidia</a></p></li><li><p><a href="#TextGenerationConfig_Ollama">Ollama</a></p></li><li><p><a href="#TextGenerationConfig_Openai">OpenAI</a></p></li><li><p><a href="#TextGenerationConfig_OpenaiCompatible">OpenAI Compatible Endpoint</a></p></li><li><p><a href="#TextGenerationConfig_Openrouter">OpenRouter</a></p></li><li><p><a href="#TextGenerationConfig_Perplexity">Perplexity</a></p></li><li><p><a href="#TextGenerationConfig_Portkey">Portkey</a></p></li><li><p><a href="#TextGenerationConfig_Together">Together</a></p></li><li><p><a href="#TextGenerationConfig_VertexaiExpress">Vertex AI Express</a></p></li><li><p><a href="#TextGenerationConfig_Xai">xAI</a></p></li><li><p><a href="#TextGenerationConfig_Xinference">XInference</a></p></li><li><p><a href="#TextGenerationConfig_ZhipuAi">ZHIPU_AI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### AI21Labs

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ai21 Labs Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Anthropic

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Anthropic Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure AI Foundry

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure AI Foundry] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry resource name.</p></td><td></td><td></td></tr><tr><td><p>[Azure AI Foundry] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry API version.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure OpenAI] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure OpenAI API version.</p></td><td><p>2024-10-21</p></td><td></td></tr><tr><td><p>[Azure OpenAI] User</p></td><td><div><div><p>String</p></div></div></td><td><p>Unique identifier representing your end-user, which can help to monitor and detect abuse.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>[Azure OpenAI] Endpoint</p></td><td><div><div><p>String</p></div></div></td><td><p>The complete endpoint for the Azure OpenAI service. You can either provide the default or a custom endpoint URL (which is used directly and overrides any parameter values), or provide the resource name and deployment ID (which are used to construct the base URL).</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure OpenAI] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>The resource name of the Azure OpenAI service used to construct the endpoint. This parameter is used if you do not provide a custom endpoint URL.</p></td><td></td><td></td></tr><tr><td><p>[Azure OpenAI] Deployment ID</p></td><td><div><div><p>String</p></div></div></td><td><p>The Deployment ID of the Azure OpenAI service used to construct the endpoint. This parameter is used if you do not provide a custom endpoint URL.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Cerebras

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Cerebras Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Cohere

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Cohere Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Databricks

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>[Databricks] Serving Endpoint Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Databricks serving endpoint name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Databricks] Serving Endpoint URL host</p></td><td><div><div><p>String</p></div></div></td><td><p>Databricks serving endpoint URL host.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### DeepInfra

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Deep Infra Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Deepseek

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Deepseek Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Docker

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Docker Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Docker Models] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Docker models base URL.</p></td><td><p><code>http://localhost:12434</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Fireworks

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Fireworks Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Gemini

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Gemini Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GitHub

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Git Hub Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GPT4All

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Gpt4all Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[GPT4ALL] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>GPT4ALL base URL.</p></td><td><p><code>http://localhost:4891/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Groq

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Groq Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Heroku AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Heroku Inference Model</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td><p><code>claude-3-7-sonnet</code></p></td><td></td></tr><tr><td><p>Heroku Inference URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Inference URL.</p></td><td><p><code>https://us.inference.heroku.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### LM Studio

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Lm Studio Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[LM Studio] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>LM Studio base URL.</p></td><td><p><code>http://localhost:1234/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Nvidia

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Nvidia Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model Name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Nvidia] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Nvidia base URL.</p></td><td><p><code>https://integrate.api.nvidia.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Ollama

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ollama Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Ollama] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Ollama base URL.</p></td><td><p><code>http://localhost:11434/api</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI model name.</p></td><td><p><code>gpt-4o-mini</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI Compatible Endpoint

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Compatible Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI compatible model name.</p></td><td><p><code>gpt-4o-mini</code></p></td><td></td></tr><tr><td><p>[OpenAI Compatible] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI compatible base URL.</p></td><td><p><code>https://server.endpoint.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenRouter

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open Router Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Perplexity

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Perplexity Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Portkey

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Portkey Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Portkey] Virtual Key</p></td><td><div><div><p>String</p></div></div></td><td><p>Portkey virtual key.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Together

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Together Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Vertex AI Express

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Vertex AI Express Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### XInference

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Inference Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Xinference] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Xinference base URL.</p></td><td><p><code>https://inference.top/api/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### ZHIPU\_AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Zhipu AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Agent\] Define Prompt Template](#AgentDefinePromptTemplate)

- [\[Chat\] Answer Prompt](#ChatAnswerPrompt)

- [\[Chat\] Completions](#ChatCompletions)

- [\[Tools\] Native Template (Reasoning only)](#ToolsNativeTemplate)

---

### Vision Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#VisionConfig_AnthropicVision">Anthropic</a></p></li><li><p><a href="#VisionConfig_AzureAiFoundryVision">Azure AI Foundry</a></p></li><li><p><a href="#VisionConfig_GeminiVision">Gemini</a></p></li><li><p><a href="#VisionConfig_GithubVision">GitHub</a></p></li><li><p><a href="#VisionConfig_GroqVision">Groq</a></p></li><li><p><a href="#VisionConfig_HuggingFaceVision">Hugging Face</a></p></li><li><p><a href="#VisionConfig_MistralaiVision">Mistral AI</a></p></li><li><p><a href="#VisionConfig_OllamaVision">Ollama</a></p></li><li><p><a href="#VisionConfig_OpenaiVision">OpenAI</a></p></li><li><p><a href="#VisionConfig_OpenrouterVision">OpenRouter</a></p></li><li><p><a href="#VisionConfig_PortkeyVision">Portkey</a></p></li><li><p><a href="#VisionConfig_VertexaiExpressVision">Vertex AI Express</a></p></li><li><p><a href="#VisionConfig_XaiVision">xAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Anthropic

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Anthropic Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure AI Foundry

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure AI Foundry Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure AI Foundry] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry resource name.</p></td><td></td><td></td></tr><tr><td><p>[Azure AI Foundry] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry API version.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Gemini

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Gemini Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GitHub

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Git Hub Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Groq

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Groq Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Ollama

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ollama Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Ollama] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Ollama base URL.</p></td><td><p><code>http://localhost:11434/api</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenRouter

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open Router Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Portkey

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Portkey Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Portkey] Virtual Key</p></td><td><div><div><p>String</p></div></div></td><td><p>Portkey virtual key.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Vertex AI Express

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Vertex AI Express Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td><p>V</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Image\] Read by (Url or Base64)](#ReadImage)

## Operations

## \[Image\] Generate (only Base64)

`<ms-inference:generate-image>`

This operation generates an image based on a prompt.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#ImageResponseAttributes">Image Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Image Generation Config](#ImageGenerationConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:IMAGE\_GENERATION\_FAILURE

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

## \[Toxicity\] Detection by Text

`<ms-inference:toxicity-detection-text>`

This operation detects harmful content in a text.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Text</p></td><td><div><div><p>Any</p></div></div></td><td><p>Text to moderate. Can be a single string or an array of strings.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr></tbody></table>

### For Configurations

- [Moderation Config](#ModerationConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOXICITY\_DETECTION\_OPERATION\_FAILURE

## \[Agent\] Define Prompt Template

`<ms-inference:agent-define-prompt-template>`

This operation defines a prompt template with instructions and data.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Template</p></td><td><div><div><p>String</p></div></div></td><td><p>Template string.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Instructions</p></td><td><div><div><p>String</p></div></div></td><td><p>Instructions for the LLM.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Data</p></td><td><div><div><p>String</p></div></div></td><td><p>Primary data content.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Chat\] Answer Prompt

`<ms-inference:chat-answer-prompt>`

This operation provides a simple chat answer for a single prompt.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Chat\] Completions

`<ms-inference:chat-completions>`

This operation provides chat completions by a messages array, including system or user messages
(conversation history).

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Messages</p></td><td><div><div><p>Array of Any</p></div></div></td><td><p>Conversation history as a JSON array.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Tools\] Native Template (Reasoning only)

`<ms-inference:tools-native-template>`

This operation defines a tools template with instructions and data. This operation is useful if you
want to create autonomous agents that can use external tools whenever a prompt can’t be answered
directly by the AI model. This operation selects which tools can be used for execution.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Template</p></td><td><div><div><p>String</p></div></div></td><td><p>Template string.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Instructions</p></td><td><div><div><p>String</p></div></div></td><td><p>Instructions for the LLM.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Data</p></td><td><div><div><p>String</p></div></div></td><td><p>Primary data content.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Tools</p></td><td><div><div><p>Array of Any</p></div></div></td><td><p>Tools configuration as a JSON array.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Image\] Read by (Url or Base64)

`<ms-inference:read-image>`

This operation reads an image from a URL or Base64 string.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Image</p></td><td><div><div><p>String</p></div></div></td><td><p>Image URL or Base64 string to send to the Vision Model.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Vision Config](#VisionConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:READ\_IMAGE\_OPERATION\_FAILURE

- MS-INFERENCE:RETRY\_EXHAUSTED

## Types

### TLS

Configures TLS to provide secure communications for the Mule app.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Enabled Protocols</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Comma-separated list of protocols enabled for this context.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Enabled Cipher Suites</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Comma-separated list of cipher suites enabled for this context.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Trust Store</strong></p></td><td><div><div><p><a href="#TrustStore">Truststore</a></p></div></div></td><td><div><div><p>Configures the TLS truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Key Store</strong></p></td><td><div><div><p><a href="#KeyStore">Keystore</a></p></div></div></td><td><div><div><p>Configures the TLS keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Revocation Check</strong></p></td><td><div><div><ul><li><p><a href="#StandardRevocationCheck">Standard Revocation Check</a></p></li><li><p><a href="#CustomOcspResponder">Custom OCSP Responder</a></p></li><li><p><a href="#CrlFile">CRL File</a></p></li></ul></div></div></td><td><div><div><p>Configures a revocation checking mechanism.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Truststore

Configures the truststore for TLS.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the truststore. Mule resolves the path relative to the current classpath and file system.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Type</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Type of truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Algorithm</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Encryption algorithm that the truststore uses.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Insecure</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>true</code>, Mule stops performing certificate validations. Setting this to <code>true</code> can make connections vulnerable to attacks.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Keystore

Configures the keystore for the TLS protocol. The keystore you generate contains a private key and a
public certificate.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the keystore. Mule resolves the path relative to the current classpath and file system.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Type</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Type of keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Alias</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Alias of the key to use when the keystore contains multiple private keys. By default, Mule uses the first key in the file.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Key Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the private key.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Algorithm</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Encryption algorithm that the keystore uses.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Standard Revocation Check

Configures standard revocation checks for TLS certificates.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Only End Entities</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Which elements to verify in the certificate chain:</p></div><div><ul><li><p><code>true</code></p><div><p>Verify only the last element in the certificate chain.</p></div></li><li><p><code>false</code></p><div><p>Verify all elements in the certificate chain.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prefer Crls</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>How to check certificate validity:</p></div><div><ul><li><p><code>true</code></p><div><p>Check the Certification Revocation List (CRL) for certificate validity.</p></div></li><li><p><code>false</code></p><div><p>Use the Online Certificate Status Protocol (OCSP) to check certificate validity.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>No Fallback</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Whether to use the secondary method to check certificate validity:</p></div><div><ul><li><p><code>true</code></p><div><p>Use the method that wasn’t specified in the <strong>Prefer Crls</strong> field (the secondary method) to check certificate validity.</p></div></li><li><p><code>false</code></p><div><p>Do not use the secondary method to check certificate validity.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Soft Fail</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>What to do if the revocation server can’t be reached or is busy:</p></div><div><ul><li><p><code>true</code></p><div><p>Avoid verification failure.</p></div></li><li><p><code>false</code></p><div><p>Allow the verification to fail.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Custom OCSP Responder

Configures a custom OCSP responder for certification revocation checks.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Url</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>URL of the OCSP responder.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Cert Alias</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Alias of the signing certificate for the OCSP response. If specified, the alias must be in the truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### CRL File

Specifies the location of the certification revocation list (CRL) file.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the CRL file.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Header

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Key</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Header name.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Value</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Header value.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr></tbody></table>

### Reconnection

Configures a reconnection strategy for an operation.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Fails Deployment</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Configures a reconnection strategy to use when a connector operation fails to connect to an external server.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Reconnection Strategy</strong></p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><div><div><p>Reconnection strategy to use.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Reconnect

Configures a standard reconnection strategy, which specifies how often to reconnect and how many
reconnection attempts the connector source or operation can make.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Frequency</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How often to attempt to reconnect, in milliseconds.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Blocking</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>false</code>, the reconnection strategy runs in a separate, non-blocking thread.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How many reconnection attempts the Mule app can make.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Reconnect Forever

Configures a forever reconnection strategy by which the connector source or operation attempts to
reconnect at a specified frequency for as long as the Mule app runs.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Frequency</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How often to attempt to reconnect, in milliseconds.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Blocking</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>false</code>, the reconnection strategy runs in a separate, non-blocking thread.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Expiration Policy

Configures an expiration policy strategy.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Max Idle Time</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Configures the maximum amount of time that a dynamic configuration instance can remain idle before Mule considers it eligible for expiration.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Time Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><div><div><p>Time unit for the <strong>Max Idle Time</strong> field.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Image Response Attributes

Configures image response attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Model</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Model.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prompt Used</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Prompt used.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Repeatable In Memory Stream

Configures the in-memory streaming strategy by which the request fails if the data exceeds the MAX
buffer size. Always run performance tests to find the optimal buffer size for your specific use
case.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Initial Buffer Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Initial amount of memory to allocate to the data stream. If the streamed data exceeds this value, the buffer expands by <strong>Buffer Size Increment</strong>, with an upper limit of <strong>Max Buffer Size</strong>.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Size Increment</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>This is by how much the buffer size expands if it exceeds its initial size. Setting a value of zero or lower means that the buffer should not expand, meaning that a <code>STREAM_MAXIMUM_SIZE_EXCEEDED</code> error is raised when the buffer gets full.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Max Buffer Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Maximum size of the buffer. If the buffer size exceeds this value, Mule raises a <code>STREAM_MAXIMUM_SIZE_EXCEEDED</code> error. A value of less than or equal to <code>0</code> means no limit.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>BYTE</p></li><li><p>KB</p></li><li><p>MB</p></li><li><p>GB</p></li></ul></div></div></td><td><div><div><p>Unit for the <strong>Initial Buffer Size</strong>, <strong>Buffer Size Increment</strong>, and <strong>Max Buffer Size</strong> fields.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Repeatable File Store Stream

Configures the repeatable file-store streaming strategy by which Mule keeps a portion of the stream
content in memory. If the stream content is larger than the configured buffer size, Mule backs up
the buffer’s content to disk and then clears the memory.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>In Memory Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Maximum amount of memory that the stream can use for data. If the amount of memory exceeds this value, Mule buffers the content to disk. To optimize performance:</p></div><div><ul><li><p>Configure a larger buffer size to avoid the number of times Mule needs to write the buffer on disk. This increases performance, but it also limits the number of concurrent requests your application can process, because it requires additional memory.</p></li><li><p>Configure a smaller buffer size to decrease memory load at the expense of response time.</p></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>BYTE</p></li><li><p>KB</p></li><li><p>MB</p></li><li><p>GB</p></li></ul></div></div></td><td><div><div><p>Unit for the <strong>In Memory Size</strong> field.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Error Mapping

Configures error mapping.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Source</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>ANY</p></li><li><p>REDELIVERY_EXHAUSTED</p></li><li><p>TRANSFORMATION</p></li><li><p>EXPRESSION</p></li><li><p>SECURITY</p></li><li><p>CLIENT_SECURITY</p></li><li><p>SERVER_SECURITY</p></li><li><p>ROUTING</p></li><li><p>CONNECTIVITY</p></li><li><p>RETRY_EXHAUSTED</p></li><li><p>TIMEOUT</p></li></ul></div></div></td><td><div><div><p>Source of the error.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Target</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Target of the error.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr></tbody></table>

### LLM Response Attributes

Configures LLM response attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Additional Attributes</strong></p></td><td><div><div><p><a href="#AdditionalAttributes">Additional Attributes</a></p></div></div></td><td><div><div><p>Additional attributes.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Token Usage</strong></p></td><td><div><div><p><a href="#TokenUsage">Token Usage</a></p></div></div></td><td><div><div><p>Token usage.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Additional Attributes

Configures additional attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Content Filter Results</strong></p></td><td><div><div><p>Object</p></div></div></td><td><div><div><p>Flags if the generated response contains potentially harmful content.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Finish Reason</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Finish reason for the LLM response.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Id</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>ID of the request.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Model</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>ID of the model used.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prompt Filter Results</strong></p></td><td><div><div><p>Array of PromptFilterResult</p></div></div></td><td><div><div><p>Flags if the user’s input contains potentially harmful content.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Token Usage

Configures token usage metadata returned as attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Input Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Number of tokens used to process the input.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Output Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Number of tokens used to generate the output.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Total Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Total number of tokens used for input and output.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Proxy

Configures a proxy for outbound connections.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Host</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Host in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Port</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Port in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Username</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Username to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Non Proxy Hosts</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Non-proxy hosts.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### NTLM Proxy

Configures an NTLM proxy for outbound connections.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Ntlm Domain</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>NTLM domain.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Host</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Host in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Port</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Port in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Username</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Username to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Non Proxy Hosts</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Non-proxy hosts.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/)

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=7f9e8892d9333944266d9cab0ded660dae019ffaabce52aba460d824e5b6c69a url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/mulesoft-inference-connector-xml-maven.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# MuleSoft Inference Connector 1.1.0 XML and Maven Support

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Although you can manually code a Mule app in XML, it is more efficient to use Anypoint Code Builder
or Studio:

- If you manually code a Mule runtime engine (Mule) app in XML, either from the Anypoint Code
  Builder or Studio XML editor, or from a text editor, you can access the connector from your app by
  adding reference statements to both your XML Mule flow and the Apache Maven `pom.xml` file.

- If you add the connector in Anypoint Code Builder or Studio, the XML is automatically populated
  the code with the connector’s namespace and schema location and it also adds a dependency to the
  `pom.xml` file.

## Add a Namespace for the Connector

Paste the following code inside the `mule` tag of the header of your configuration XML:

```xml
http://www.mulesoft.org/schema/mule/mulesoft-inference
http://www.mulesoft.org/schema/mule/mulesoft-inference/current/mule-mulesoft-inference.xsd
```

This example shows how the namespace statements are placed in the `<mule>` XML block:

```xml
<mule xmlns:mulesoft-inference="http://www.mulesoft.org/schema/mule/mulesoft-inference"
	xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
	xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns:file="http://www.mulesoft.org/schema/mule/file"
	xmlns="http://www.mulesoft.org/schema/mule/core"
	xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core
	http://www.mulesoft.org/schema/mule/core/current/mule.xsd
	http://www.mulesoft.org/schema/mule/file
	http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
	http://www.mulesoft.org/schema/mule/http
	http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
	http://www.mulesoft.org/schema/mule/ee/core
	http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd
	http://www.mulesoft.org/schema/mule/mulesoft-inference
	http://www.mulesoft.org/schema/mule/mulesoft-inference/current/mule-mulesoft-inference.xsd">
```

## Add a POM File Dependency

The Apache Maven `pom.xml` file generated by Anypoint Studio contains dependencies for a Mule app.
If you code a Mule app manually, include this XML snippet in your `pom.xml` file to enable access to
this connector:

```xml
<dependency>
  <groupId>com.mulesoft.connectors</groupId>
  <artifactId>mule4-inference-connector</artifactId>
  <version>x.x.x</version>
  <classifier>mule-plugin</classifier>
</dependency>
```

Replace `x.x.x` with the version that corresponds to the connector you are using.

To obtain the most up-to-date `pom.xml` file information:

1.  Go to [Anypoint Exchange](https://www.mulesoft.com/exchange/).

2.  In Exchange, click **Login** and supply your Anypoint Platform username and password.

3.  In Exchange, search for `inference`.

4.  Select the connector.

5.  Click **Dependency Snippets** near the upper right of the screen.

## Next Step

After completing your namespace and pom.xml file, you can try the
[Examples](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-examples).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=6e3be76415d9e6d466987feddae9e9a128bb40743bdb945af5eac4cba4657ac9 url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/index.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# MuleSoft Inference Connector 1.2

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Anypoint Connector for MuleSoft Inference (MuleSoft Inference Connector) provides access to
inference offerings for large language models (LLMs) from multiple providers including OpenAI,
OpenAI Compatible Endpoints, OpenRouter, Heroku AI, Azure AI Foundry, Azure OpenAI, and many others.
This connector provides operations to interface directly with the API of various inference
providers, enabling seamless integration of AI capabilities into your Mule applications.

For information about compatibility and fixed issues, see the MuleSoft Inference Connector release
notes.

## Before You Begin

To use this connector, you must be familiar with:

- Anypoint Connectors

- Mule runtime engine (Mule)

- Elements and global elements in a Mule flow

- How to create a Mule app using Anypoint Code Builder or Anypoint Studio

Before creating an app, you must have:

- Java 17 (required for compilation and runtime)

- Apache Maven

- Credentials to access the MuleSoft Inference Connector target resource

- Anypoint Platform

- The latest versions of Anypoint Code Builder or Anypoint Studio

## Key Features

MuleSoft Inference Connector simplifies AI integration into Mule applications with:

- **Seamless Interaction with Inferenced LLMs**

  Integrate and utilize large language models (LLMs) effortlessly for tasks such as natural language
  processing, text generation, and more, including advanced features such as image analysis or
  generation, and toxicity detection in both user-provided and LLM-generated text.

- **Optimized Performance**

  Deliver high efficiency and performance in enterprise-grade Mule apps, ensuring smooth handling of
  AI operations.

- **Secure Integration**

  Connect corporate applications and data with innovative new AI offerings without damaging security
  posture. Provide full [TLS](https://docs.mulesoft.com/mule-runtime/latest/tls-configuration) support for secure Mule
  apps.

## Supported Inference Providers

- AI21Labs (beta)

- Anthropic (beta)

- Azure AI Foundry

- Azure OpenAI

- Cerebras (beta)

- Cohere (beta)

- Databricks (beta)

- DeepInfra (beta)

- DeepSeek (beta)

- Docker Models (beta)

- Fireworks (beta)

- Gemini (beta)

- GitHub Models (beta)

- GPT4ALL (beta)

- Groq AI (beta)

- Heroku AI

- Hugging Face (beta) (`hf-inference` only)

- LM Studio (beta)

- Mistral AI (beta)

- NVIDIA NIM (beta)

- Ollama (beta)

- OpenAI

- OpenAI Compatible Endpoints

- OpenRouter

- Perplexity (beta)

- Portkey (beta)

- Swisscom (beta)

- Together.ai (beta)

- Vertex AI Express (beta)

- XAI (beta)

- Xinference (beta)

- ZHIPU AI (beta)

## Supported Moderation Providers

- Mistral AI (beta)

- OpenAI

## Supported Vision Model Providers

- Anthropic (beta)

- Azure AI Foundry

- Gemini (beta) (`Base64` only)

- GitHub Models (beta)

- Groq AI (beta)

- Hugging Face (beta) (`hf-inference` only)

- Mistral AI (beta)

- Ollama (beta)

- OpenAI

- OpenRouter

- Portkey (beta)

- Vertex AI Express (beta)

- XAI (beta)

## Supported Image Generation Providers

- Heroku AI

- Hugging Face (beta) (`hf-inference` only)

- OpenAI

- Stability AI (beta)

- XAI (beta)

> [!NOTE] To keep pace with the rapidly evolving AI landscape, certain LLMs are marked as beta.
> These are early-stage integrations that may change based on stability, demand, or provider
> updates. You can explore them but should do so with awareness that support might be limited and
> subject to change.

## Supported Operations by Offering

Not all operations are supported by each provider. This table provides a detailed view of operation
support across all providers.

<table><colgroup><col> <col> <col> <col> <col> <col> <col> <col> <col></colgroup><thead><tr><th>LLM Provider</th><th>[Chat] Answer Prompt</th><th>[Chat] Completions</th><th>[Agent] Define Prompt Template</th><th>[Tools] Native Template (Reasoning only)*</th><th>[Image] Read by (Url or Base64)*</th><th>[Image] Generate (only Base64)*</th><th>[Toxicity] Detection by Text*</th><th>[MCP] Tooling</th></tr></thead><tbody><tr><td><p>AI21Labs</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Anthropic</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Azure AI Foundry</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Azure OpenAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Cerebras</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Cohere</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Databricks</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>DeepInfra</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>DeepSeek</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Docker Models</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Fireworks</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Gemini</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Base64 only</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>GitHub Models</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>GPT4ALL</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Groq AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Heroku AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Hugging Face</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>LM Studio</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Mistral AI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>Yes</p></td></tr><tr><td><p>NVIDIA NIM</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Ollama</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>OpenAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td></tr><tr><td><p>OpenAI Compatible Endpoints</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>OpenRouter</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Perplexity</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Portkey</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Stability AI</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>Swisscom</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Together.ai</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Vertex AI Express</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td></tr><tr><td><p>XAI</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Xinference</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr><tr><td><p>Zhipu</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>Yes</p></td><td><p>No</p></td><td><p>No</p></td><td><p>No</p></td><td><p>Yes</p></td></tr></tbody></table>

> [!NOTE] `*` indicates that support depends on the specific model used.

## Next Step

After you complete the prerequisites, you are ready to create an app and configure the connector
using [Anypoint Studio](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-studio) or
[Anypoint Code Builder](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-config-acb).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Using Anypoint Code Builder to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-acb)

- [Using Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio)

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-additional-config)

- [MuleSoft Inference Connector Examples](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-examples)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=42185dded4c288a3565a3590fd6aee9c2ce259eb72c8e62e283914b77b7297a5 url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# MuleSoft Inference Connector 1.2 Reference

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

MuleSoft Inference Connector provides operations to interface directly with the API of various
inference providers, enabling seamless integration of AI capabilities into your Mule applications.

## Configurations

---

### Image Generation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#ImageGenerationConfig_HerokuImage">Heroku AI</a></p></li><li><p><a href="#ImageGenerationConfig_HuggingFaceImage">Hugging Face</a></p></li><li><p><a href="#ImageGenerationConfig_OpenaiImage">OpenAI</a></p></li><li><p><a href="#ImageGenerationConfig_StabilityAiImage">Stability AI</a></p></li><li><p><a href="#ImageGenerationConfig_XaiImage">xAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Heroku AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Heroku Inference Model</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Heroku Diffusion URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Heroku diffusion URL.</p></td><td><p><code>https://us.inference.heroku.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Stability AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Stability AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Image\] Generate (only Base64)](#GenerateImage)

---

### Moderation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#ModerationConfig_MistralaiModeration">Mistral AI</a></p></li><li><p><a href="#ModerationConfig_OpenaiModeration">OpenAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Toxicity\] Detection by Text](#ToxicityDetectionText)

---

### Text Generation Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#TextGenerationConfig_Ai21labs">AI21Labs</a></p></li><li><p><a href="#TextGenerationConfig_Anthropic">Anthropic</a></p></li><li><p><a href="#TextGenerationConfig_AzureAiFoundry">Azure AI Foundry</a></p></li><li><p><a href="#TextGenerationConfig_AzureOpenai">Azure OpenAI</a></p></li><li><p><a href="#TextGenerationConfig_Cerebras">Cerebras</a></p></li><li><p><a href="#TextGenerationConfig_Cohere">Cohere</a></p></li><li><p><a href="#TextGenerationConfig_Databricks">Databricks</a></p></li><li><p><a href="#TextGenerationConfig_Deepinfra">DeepInfra</a></p></li><li><p><a href="#TextGenerationConfig_Deepseek">Deepseek</a></p></li><li><p><a href="#TextGenerationConfig_Docker">Docker</a></p></li><li><p><a href="#TextGenerationConfig_Fireworks">Fireworks</a></p></li><li><p><a href="#TextGenerationConfig_Gemini">Gemini</a></p></li><li><p><a href="#TextGenerationConfig_Github">GitHub</a></p></li><li><p><a href="#TextGenerationConfig_Gpt4all">GPT4All</a></p></li><li><p><a href="#TextGenerationConfig_Groq">Groq</a></p></li><li><p><a href="#TextGenerationConfig_Heroku">Heroku AI</a></p></li><li><p><a href="#TextGenerationConfig_HuggingFace">Hugging Face</a></p></li><li><p><a href="#TextGenerationConfig_Lmstudio">LM Studio</a></p></li><li><p><a href="#TextGenerationConfig_Mistralai">Mistral AI</a></p></li><li><p><a href="#TextGenerationConfig_Nvidia">NVIDIA</a></p></li><li><p><a href="#TextGenerationConfig_Ollama">Ollama</a></p></li><li><p><a href="#TextGenerationConfig_Openai">OpenAI</a></p></li><li><p><a href="#TextGenerationConfig_OpenaiCompatible">OpenAI Compatible Endpoint</a></p></li><li><p><a href="#TextGenerationConfig_Openrouter">OpenRouter</a></p></li><li><p><a href="#TextGenerationConfig_Perplexity">Perplexity</a></p></li><li><p><a href="#TextGenerationConfig_Portkey">Portkey</a></p></li><li><p><a href="#TextGenerationConfig_Swisscom">Swisscom</a></p></li><li><p><a href="#TextGenerationConfig_Together">Together</a></p></li><li><p><a href="#TextGenerationConfig_VertexaiExpress">Vertex AI Express</a></p></li><li><p><a href="#TextGenerationConfig_Xai">xAI</a></p></li><li><p><a href="#TextGenerationConfig_Xinference">XInference</a></p></li><li><p><a href="#TextGenerationConfig_ZhipuAi">ZHIPU_AI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### AI21Labs

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ai21 Labs Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Anthropic

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Anthropic Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure AI Foundry

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure AI Foundry] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry resource name.</p></td><td></td><td></td></tr><tr><td><p>[Azure AI Foundry] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry API version.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure OpenAI] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure OpenAI API version.</p></td><td><p>2024-10-21</p></td><td></td></tr><tr><td><p>[Azure OpenAI] User</p></td><td><div><div><p>String</p></div></div></td><td><p>Unique identifier representing your end-user, which can help to monitor and detect abuse.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>[Azure OpenAI] Endpoint</p></td><td><div><div><p>String</p></div></div></td><td><p>The complete endpoint for the Azure OpenAI service. You can either provide the default or a custom endpoint URL (which is used directly and overrides any parameter values), or provide the resource name and deployment ID (which are used to construct the base URL).</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure OpenAI] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>The resource name of the Azure OpenAI service used to construct the endpoint. This parameter is used if you do not provide a custom endpoint URL.</p></td><td></td><td></td></tr><tr><td><p>[Azure OpenAI] Deployment ID</p></td><td><div><div><p>String</p></div></div></td><td><p>The Deployment ID of the Azure OpenAI service used to construct the endpoint. This parameter is used if you do not provide a custom endpoint URL.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Cerebras

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Cerebras Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Cohere

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Cohere Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Databricks

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>[Databricks] Serving Endpoint Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Databricks serving endpoint name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Databricks] Serving Endpoint URL host</p></td><td><div><div><p>String</p></div></div></td><td><p>Databricks serving endpoint URL host.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### DeepInfra

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Deep Infra Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Deepseek

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Deepseek Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Docker

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Docker Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Docker Models] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Docker models base URL.</p></td><td><p><code>http://localhost:12434</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Fireworks

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Fireworks Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Gemini

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Gemini Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GitHub

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Git Hub Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GPT4All

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Gpt4all Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[GPT4ALL] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>GPT4ALL base URL.</p></td><td><p><code>http://localhost:4891/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Groq

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Groq Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Heroku AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Heroku Inference Model</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td><p><code>claude-3-7-sonnet</code></p></td><td></td></tr><tr><td><p>Heroku Inference URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Inference URL.</p></td><td><p><code>https://us.inference.heroku.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### LM Studio

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Lm Studio Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[LM Studio] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>LM Studio base URL.</p></td><td><p><code>http://localhost:1234/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### NVIDIA

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>NVIDIA Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model Name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>NVIDIA Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>NVIDIA base URL.</p></td><td><p><code>https://integrate.api.nvidia.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Ollama

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ollama Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Ollama] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Ollama base URL.</p></td><td><p><code>http://localhost:11434/api</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI model name.</p></td><td><p><code>gpt-4o-mini</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI Compatible Endpoint

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Compatible Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI compatible model name.</p></td><td><p><code>gpt-4o-mini</code></p></td><td></td></tr><tr><td><p>[OpenAI Compatible] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>OpenAI compatible base URL.</p></td><td><p><code>https://server.endpoint.com</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenRouter

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open Router Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Perplexity

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Perplexity Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Portkey

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Portkey Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Portkey] Virtual Key</p></td><td><div><div><p>String</p></div></div></td><td><p>Portkey virtual key.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Swisscom

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Swisscom Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td><p><code>meta/llama-3.3-70b-instruct</code></p></td><td></td></tr><tr><td><p>[Swisscom] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Swisscom base URL.</p></td><td><p><code>https://api.swisscom.com/layer/swiss-ai-platform/{modelName}/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Together

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Together Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Vertex AI Express

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Vertex AI Express Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### XInference

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Inference Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Xinference] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Xinference base URL.</p></td><td><p><code>https://inference.top/api/v1</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### ZHIPU\_AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Zhipu AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Agent\] Define Prompt Template](#AgentDefinePromptTemplate)

- [\[Chat\] Answer Prompt](#ChatAnswerPrompt)

- [\[Chat\] Completions](#ChatCompletions)

- [\[MCP\] Tooling](#McpToolsNativeTemplate)

- [\[Tools\] Native Template (Reasoning only)](#ToolsNativeTemplate)

---

### Vision Config

#### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p>String</p></td><td><p>Name for this configuration. Connectors reference the configuration with this name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Connection</p></td><td><div><div><ul><li><p><a href="#VisionConfig_AnthropicVision">Anthropic</a></p></li><li><p><a href="#VisionConfig_AzureAiFoundryVision">Azure AI Foundry</a></p></li><li><p><a href="#VisionConfig_GeminiVision">Gemini</a></p></li><li><p><a href="#VisionConfig_GithubVision">GitHub</a></p></li><li><p><a href="#VisionConfig_GroqVision">Groq</a></p></li><li><p><a href="#VisionConfig_HuggingFaceVision">Hugging Face</a></p></li><li><p><a href="#VisionConfig_MistralaiVision">Mistral AI</a></p></li><li><p><a href="#VisionConfig_OllamaVision">Ollama</a></p></li><li><p><a href="#VisionConfig_OpenaiVision">OpenAI</a></p></li><li><p><a href="#VisionConfig_OpenrouterVision">OpenRouter</a></p></li><li><p><a href="#VisionConfig_PortkeyVision">Portkey</a></p></li><li><p><a href="#VisionConfig_VertexaiExpressVision">Vertex AI Express</a></p></li><li><p><a href="#VisionConfig_XaiVision">xAI</a></p></li></ul></div></div></td><td><p>Connection types for this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Name</p></td><td><div><div><p>String</p></div></div></td><td><p>ID used to reference this configuration.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Expiration Policy</p></td><td><div><div><p><a href="#ExpirationPolicy">Expiration Policy</a></p></div></div></td><td><p>Configures an expiration policy for the configuration.</p></td><td></td><td></td></tr></tbody></table>

#### Connection Types

##### Anthropic

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Anthropic Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Azure AI Foundry

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Azure AI Foundry Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Azure AI Foundry] Resource Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry resource name.</p></td><td></td><td></td></tr><tr><td><p>[Azure AI Foundry] API Version</p></td><td><div><div><p>String</p></div></div></td><td><p>Azure AI Foundry API version.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Gemini

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Gemini Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### GitHub

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Git Hub Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Groq

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Groq Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Hugging Face

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Hugging Face Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Mistral AI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Mistral AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Ollama

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Ollama Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Ollama] Base URL</p></td><td><div><div><p>String</p></div></div></td><td><p>Ollama base URL.</p></td><td><p><code>http://localhost:11434/api</code></p></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open AI Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### OpenRouter

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Open Router Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Portkey

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Portkey Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>[Portkey] Virtual Key</p></td><td><div><div><p>String</p></div></div></td><td><p>Portkey virtual key.</p></td><td></td><td></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### Vertex AI Express

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>Vertex AI Express Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

##### xAI

###### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Proxy Configuration</p></td><td><div><div><p>One of:</p></div><div><ul><li><p><a href="#Proxy">Proxy</a></p></li><li><p><a href="#NtlmProxy">NTLM Proxy</a></p></li></ul></div></div></td><td><p>Configures a proxy for outbound connections.</p></td><td></td><td></td></tr><tr><td><p>X Ai Model Name</p></td><td><div><div><p>String</p></div></div></td><td><p>Model name.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>TLS Configuration</p></td><td><div><div><p><a href="#Tls">TLS</a></p></div></div></td><td><p>If HTTPS is configured as a protocol, then you must configure at least the keystore configuration.</p></td><td></td><td></td></tr><tr><td><p>API Key</p></td><td><div><div><p>String</p></div></div></td><td><p>API key as required by the inference provider.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Custom headers</p></td><td><div><div><p>Array of <a href="#DefaultHeader">Header</a></p></div></div></td><td><p>Custom headers are additional HTTP headers to include with each request. For any existing header key, the custom value will take precedence and overwrite the existing one.</p></td><td></td><td><p>V</p></td></tr><tr><td><p>Timeout</p></td><td><div><div><p>Number</p></div></div></td><td><p>Response timeout value set for each inference HTTP request.</p></td><td><p>60</p></td><td></td></tr><tr><td><p>Timeout Unit</p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><p>Response timeout unit for the <strong>Timeout</strong> field.</p></td><td><p>SECONDS</p></td><td></td></tr><tr><td><p>Max Tokens</p></td><td><div><div><p>Number</p></div></div></td><td><p>Defines the number of LLM tokens to use when generating a response. This field helps control the usage and costs when engaging with LLMs.</p></td><td><p>500</p></td><td></td></tr><tr><td><p>Temperature</p></td><td><div><div><p>Number</p></div></div></td><td><p>Number between 0 and 2. The temperature is used to control the randomness of the output. When you set it higher, you’ll get more random outputs. When you set it lower (towards 0), the values are more deterministic.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Top P</p></td><td><div><div><p>Number</p></div></div></td><td><p>Controls diversity by creating a nucleus of the most probable words to choose from for the next token. Specifies the cumulative probability score threshold that the tokens must reach.</p></td><td><p>0.9</p></td><td></td></tr><tr><td><p>Reconnection</p></td><td><div><div><p><a href="#Reconnection">Reconnection</a></p></div></div></td><td><p>When the application is deployed, a connectivity test is performed on all connectors. If set to <code>true</code>, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.</p></td><td></td><td></td></tr></tbody></table>

## Supported Operations

- [\[Image\] Read by (Url or Base64)](#ReadImage)

## Operations

## \[Image\] Generate (only Base64)

`<ms-inference:generate-image>`

This operation generates image based on the text prompt using AI image generation models.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s text prompt describing the desired image.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#ImageResponseAttributes">Image Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Image Generation Config](#ImageGenerationConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:IMAGE\_GENERATION\_FAILURE

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

## \[Toxicity\] Detection by Text

`<ms-inference:toxicity-detection-text>`

This operation detects harmful content in a text.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Text</p></td><td><div><div><p>Any</p></div></div></td><td><p>Text to moderate. Can be a single string or an array of strings.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr></tbody></table>

### For Configurations

- [Moderation Config](#ModerationConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOXICITY\_DETECTION\_OPERATION\_FAILURE

## \[Agent\] Define Prompt Template

`<ms-inference:agent-define-prompt-template>`

This operation defines a prompt template with instructions and data.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Template</p></td><td><div><div><p>String</p></div></div></td><td><p>Template string.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Instructions</p></td><td><div><div><p>String</p></div></div></td><td><p>Instructions for the LLM.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Data</p></td><td><div><div><p>String</p></div></div></td><td><p>Primary data content.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:MCP\_SERVER\_ERROR

- MS-INFERENCE:MCP\_TOOLS\_OPERATION\_FAILURE

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Chat\] Answer Prompt

`<ms-inference:chat-answer-prompt>`

This operation provides a simple chat answer for a single prompt.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:MCP\_SERVER\_ERROR

- MS-INFERENCE:MCP\_TOOLS\_OPERATION\_FAILURE

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Chat\] Completions

`<ms-inference:chat-completions>`

This operation provides chat completions by a messages array, including system or user messages
(conversation history).

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Messages</p></td><td><div><div><p>Array of Any</p></div></div></td><td><p>Conversation history as a JSON array.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:MCP\_SERVER\_ERROR

- MS-INFERENCE:MCP\_TOOLS\_OPERATION\_FAILURE

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[MCP\] Tooling

`<ms-inference:mcp-tools-native-template>`

This operation executes MCP tools using a defined template that includes instructions, data, and
tools fetched directly from MCP servers.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Mcp Config References</p></td><td><div><div><p>Array of <a href="#McpConfig">Mcp Config</a></p></div></div></td><td><p>List of MCP client configuration references.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Template</p></td><td><div><div><p>String</p></div></div></td><td><p>Template string.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Instructions</p></td><td><div><div><p>String</p></div></div></td><td><p>Instructions for the LLM.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Data</p></td><td><div><div><p>String</p></div></div></td><td><p>Primary data content.</p></td><td><p>#[payload]</p></td><td></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:MCP\_SERVER\_ERROR

- MS-INFERENCE:MCP\_TOOLS\_OPERATION\_FAILURE

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Tools\] Native Template (Reasoning only)

`<ms-inference:tools-native-template>`

This operation defines a tools template with instructions and data. This operation is useful if you
want to create autonomous agents that can use external tools whenever a prompt can’t be answered
directly by the AI model. This operation selects which tools can be used for execution.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Template</p></td><td><div><div><p>String</p></div></div></td><td><p>Template string.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Instructions</p></td><td><div><div><p>String</p></div></div></td><td><p>Instructions for the LLM.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Data</p></td><td><div><div><p>String</p></div></div></td><td><p>Primary data content.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Tools</p></td><td><div><div><p>Array of Any</p></div></div></td><td><p>Tools configuration as a JSON array.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Text Generation Config](#TextGenerationConfig)

### Throws

- MS-INFERENCE:CHAT\_OPERATION\_FAILURE

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:MCP\_SERVER\_ERROR

- MS-INFERENCE:MCP\_TOOLS\_OPERATION\_FAILURE

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:RETRY\_EXHAUSTED

- MS-INFERENCE:TOOLS\_OPERATION\_FAILURE

## \[Image\] Read by (Url or Base64)

`<ms-inference:read-image>`

This operation reads an image from a URL or Base64 string.

### Parameters

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Name</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p>Configuration</p></td><td><p>String</p></td><td><p>Name of the configuration to use.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Prompt</p></td><td><div><div><p>String</p></div></div></td><td><p>User’s prompt.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Image</p></td><td><div><div><p>String</p></div></div></td><td><p>Image URL or Base64 string to send to the Vision Model.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Additional Request Attributes</p></td><td><div><div><p>Any</p></div></div></td><td><p>Additional attributes in the request payload to generate more relevant and precise LLM outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Mime Type</p></td><td><div><div><p>String</p></div></div></td><td><p>MIME type of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Output Encoding</p></td><td><div><div><p>String</p></div></div></td><td><p>Encoding of the payload that this operation outputs.</p></td><td></td><td></td></tr><tr><td><p>Config Ref</p></td><td><div><div><p>ConfigurationProvider</p></div></div></td><td><p>Name of the configuration to use to execute this component.</p></td><td></td><td><p>x</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><div><div><ul><li><p><a href="#RepeatableInMemoryStream">Repeatable In Memory Stream</a></p></li><li><p><a href="#RepeatableFileStoreStream">Repeatable File Store Stream</a></p></li><li><p>non-repeatable-stream</p></li></ul></div></div></td><td><p>Configures how Mule processes streams. Repeatable streams are the default behavior.</p></td><td></td><td></td></tr><tr><td><p>Target Variable</p></td><td><div><div><p>String</p></div></div></td><td><p>Name of the variable that stores the operation’s output.</p></td><td></td><td></td></tr><tr><td><p>Target Value</p></td><td><div><div><p>String</p></div></div></td><td><p>Expression that evaluates the operation’s output. The outcome of the expression is stored in the <strong>Target Variable</strong> field.</p></td><td><p><code>#[payload]</code></p></td><td></td></tr><tr><td><p>Error Mappings</p></td><td><div><div><p>Array of <a href="#ErrorMapping">Error Mapping</a></p></div></div></td><td><p>Set of error mappings.</p></td><td></td><td></td></tr><tr><td><p>Reconnection Strategy</p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><p>Retry strategy in case of connectivity errors.</p></td><td></td><td></td></tr></tbody></table>

### Output

<table><colgroup><col> <col></colgroup><tbody><tr><td><p>Type</p></td><td><p>Any</p></td></tr><tr><td><p>Attributes Type</p></td><td><div><div><p><a href="#LlmResponseAttributes">LLM Response Attributes</a></p></div></div></td></tr></tbody></table>

### For Configurations

- [Vision Config](#VisionConfig)

### Throws

- MS-INFERENCE:CONNECTIVITY

- MS-INFERENCE:INVALID\_CONNECTION

- MS-INFERENCE:INVALID\_PROVIDER

- MS-INFERENCE:RATE\_LIMIT\_EXCEEDED

- MS-INFERENCE:READ\_IMAGE\_OPERATION\_FAILURE

- MS-INFERENCE:RETRY\_EXHAUSTED

## Types

### TLS

Configures TLS to provide secure communications for the Mule app.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Enabled Protocols</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Comma-separated list of protocols enabled for this context.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Enabled Cipher Suites</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Comma-separated list of cipher suites enabled for this context.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Trust Store</strong></p></td><td><div><div><p><a href="#TrustStore">Truststore</a></p></div></div></td><td><div><div><p>Configures the TLS truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Key Store</strong></p></td><td><div><div><p><a href="#KeyStore">Keystore</a></p></div></div></td><td><div><div><p>Configures the TLS keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Revocation Check</strong></p></td><td><div><div><ul><li><p><a href="#StandardRevocationCheck">Standard Revocation Check</a></p></li><li><p><a href="#CustomOcspResponder">Custom OCSP Responder</a></p></li><li><p><a href="#CrlFile">CRL File</a></p></li></ul></div></div></td><td><div><div><p>Configures a revocation checking mechanism.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Truststore

Configures the truststore for TLS.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the truststore. Mule resolves the path relative to the current classpath and file system.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Type</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Type of truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Algorithm</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Encryption algorithm that the truststore uses.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Insecure</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>true</code>, Mule stops performing certificate validations. Setting this to <code>true</code> can make connections vulnerable to attacks.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Keystore

Configures the keystore for the TLS protocol. The keystore you generate contains a private key and a
public certificate.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the keystore. Mule resolves the path relative to the current classpath and file system.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Type</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Type of keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Alias</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Alias of the key to use when the keystore contains multiple private keys. By default, Mule uses the first key in the file.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Key Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the private key.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password used to protect the keystore.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Algorithm</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Encryption algorithm that the keystore uses.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Standard Revocation Check

Configures standard revocation checks for TLS certificates.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Only End Entities</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Which elements to verify in the certificate chain:</p></div><div><ul><li><p><code>true</code></p><div><p>Verify only the last element in the certificate chain.</p></div></li><li><p><code>false</code></p><div><p>Verify all elements in the certificate chain.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prefer Crls</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>How to check certificate validity:</p></div><div><ul><li><p><code>true</code></p><div><p>Check the Certification Revocation List (CRL) for certificate validity.</p></div></li><li><p><code>false</code></p><div><p>Use the Online Certificate Status Protocol (OCSP) to check certificate validity.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>No Fallback</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Whether to use the secondary method to check certificate validity:</p></div><div><ul><li><p><code>true</code></p><div><p>Use the method that wasn’t specified in the <strong>Prefer Crls</strong> field (the secondary method) to check certificate validity.</p></div></li><li><p><code>false</code></p><div><p>Do not use the secondary method to check certificate validity.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Soft Fail</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>What to do if the revocation server can’t be reached or is busy:</p></div><div><ul><li><p><code>true</code></p><div><p>Avoid verification failure.</p></div></li><li><p><code>false</code></p><div><p>Allow the verification to fail.</p></div></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Custom OCSP Responder

Configures a custom OCSP responder for certification revocation checks.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Url</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>URL of the OCSP responder.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Cert Alias</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Alias of the signing certificate for the OCSP response. If specified, the alias must be in the truststore.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### CRL File

Specifies the location of the certification revocation list (CRL) file.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Path</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Path to the CRL file.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Header

Represents an HTTP header.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Key</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Header name.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Value</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Header value.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr></tbody></table>

### Reconnection

Configures a reconnection strategy for an operation.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Fails Deployment</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>Configures a reconnection strategy to use when a connector operation fails to connect to an external server.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Reconnection Strategy</strong></p></td><td><div><div><ul><li><p><a href="#Reconnect">Reconnect</a></p></li><li><p><a href="#ReconnectForever">Reconnect Forever</a></p></li></ul></div></div></td><td><div><div><p>Reconnection strategy to use.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Reconnect

Configures a standard reconnection strategy, which specifies how often to reconnect and how many
reconnection attempts the connector source or operation can make.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Frequency</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How often to attempt to reconnect, in milliseconds.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Blocking</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>false</code>, the reconnection strategy runs in a separate, non-blocking thread.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How many reconnection attempts the Mule app can make.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Reconnect Forever

Configures a forever reconnection strategy by which the connector source or operation attempts to
reconnect at a specified frequency for as long as the Mule app runs.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Frequency</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>How often to attempt to reconnect, in milliseconds.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Blocking</strong></p></td><td><div><div><p>Boolean</p></div></div></td><td><div><div><p>If <code>false</code>, the reconnection strategy runs in a separate, non-blocking thread.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Expiration Policy

Configures an expiration policy strategy.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Max Idle Time</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Configures the maximum amount of time that a dynamic configuration instance can remain idle before Mule considers it eligible for expiration.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Time Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>NANOSECONDS</p></li><li><p>MICROSECONDS</p></li><li><p>MILLISECONDS</p></li><li><p>SECONDS</p></li><li><p>MINUTES</p></li><li><p>HOURS</p></li><li><p>DAYS</p></li></ul></div></div></td><td><div><div><p>Time unit for the <strong>Max Idle Time</strong> field.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Image Response Attributes

Configures image response attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Model</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Model.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prompt Used</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Prompt used.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Repeatable In Memory Stream

Configures the in-memory streaming strategy by which the request fails if the data exceeds the MAX
buffer size. Always run performance tests to find the optimal buffer size for your specific use
case.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Initial Buffer Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Initial amount of memory to allocate to the data stream. If the streamed data exceeds this value, the buffer expands by <strong>Buffer Size Increment</strong>, with an upper limit of <strong>Max Buffer Size</strong>.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Size Increment</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>This is by how much the buffer size expands if it exceeds its initial size. Setting a value of zero or lower means that the buffer should not expand, meaning that a <code>STREAM_MAXIMUM_SIZE_EXCEEDED</code> error is raised when the buffer gets full.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Max Buffer Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Maximum size of the buffer. If the buffer size exceeds this value, Mule raises a <code>STREAM_MAXIMUM_SIZE_EXCEEDED</code> error. A value of less than or equal to <code>0</code> means no limit.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>BYTE</p></li><li><p>KB</p></li><li><p>MB</p></li><li><p>GB</p></li></ul></div></div></td><td><div><div><p>Unit for the <strong>Initial Buffer Size</strong>, <strong>Buffer Size Increment</strong>, and <strong>Max Buffer Size</strong> fields.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Repeatable File Store Stream

Configures the repeatable file-store streaming strategy by which Mule keeps a portion of the stream
content in memory. If the stream content is larger than the configured buffer size, Mule backs up
the buffer’s content to disk and then clears the memory.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>In Memory Size</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Maximum amount of memory that the stream can use for data. If the amount of memory exceeds this value, Mule buffers the content to disk. To optimize performance:</p></div><div><ul><li><p>Configure a larger buffer size to avoid the number of times Mule needs to write the buffer on disk. This increases performance, but it also limits the number of concurrent requests your application can process, because it requires additional memory.</p></li><li><p>Configure a smaller buffer size to decrease memory load at the expense of response time.</p></li></ul></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Buffer Unit</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>BYTE</p></li><li><p>KB</p></li><li><p>MB</p></li><li><p>GB</p></li></ul></div></div></td><td><div><div><p>Unit for the <strong>In Memory Size</strong> field.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Error Mapping

Configures error mapping.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Source</strong></p></td><td><div><div><p>Enumeration, one of:</p></div><div><ul><li><p>ANY</p></li><li><p>REDELIVERY_EXHAUSTED</p></li><li><p>TRANSFORMATION</p></li><li><p>EXPRESSION</p></li><li><p>SECURITY</p></li><li><p>CLIENT_SECURITY</p></li><li><p>SERVER_SECURITY</p></li><li><p>ROUTING</p></li><li><p>CONNECTIVITY</p></li><li><p>RETRY_EXHAUSTED</p></li><li><p>TIMEOUT</p></li></ul></div></div></td><td><div><div><p>Source of the error.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Target</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Target of the error.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr></tbody></table>

### LLM Response Attributes

Configures LLM response attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Additional Attributes</strong></p></td><td><div><div><p><a href="#AdditionalAttributes">Additional Attributes</a></p></div></div></td><td><div><div><p>Additional attributes.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Token Usage</strong></p></td><td><div><div><p><a href="#TokenUsage">Token Usage</a></p></div></div></td><td><div><div><p>Token usage.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Additional Attributes

Configures additional attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Content Filter Results</strong></p></td><td><div><div><p>Object</p></div></div></td><td><div><div><p>Flags if the generated response contains potentially harmful content.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Finish Reason</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Finish reason for the LLM response.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Id</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>ID of the request.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Model</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>ID of the model used.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prompt Filter Results</strong></p></td><td><div><div><p>Array of <a href="#PromptFilterResult">Prompt Filter Result</a></p></div></div></td><td><div><div><p>Flags if the user’s input contains potentially harmful content.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Prompt Filter Result

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Content Filter Results</strong></p></td><td><div><div><p>Object</p></div></div></td><td><div><div><p>Flags if the generated response contains potentially harmful content.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Prompt Index</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Sequence of prompts sent to the LLM.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Token Usage

Configures token usage metadata returned as attributes.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Input Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Number of tokens used to process the input.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Output Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Number of tokens used to generate the output.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Total Count</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Total number of tokens used for input and output.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### Mcp Config

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Mcp Client Config Ref</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Reference to the MCP client configuration defined as part of configuration from MCP Connector. This value must match the name of a <code>mcp:client-config</code> declared in the global configuration. It is used to discover and execute tools against the targeted MCP server.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr></tbody></table>

### Proxy

Configures a basic HTTP Proxy based on host and port, and optionally a username and password for
proxy authentication.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Host</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Host in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Port</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Port in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Username</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Username to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Non Proxy Hosts</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>List of comma separated hosts against which the proxy should not be used.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

### NTLM Proxy

Configures an NTLM proxy for outbound connections.

<table><colgroup><col> <col> <col> <col> <col></colgroup><thead><tr><th>Field</th><th>Type</th><th>Description</th><th>Default Value</th><th>Required</th></tr></thead><tbody><tr><td><p><strong>Ntlm Domain</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Domain to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Host</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Host in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Port</strong></p></td><td><div><div><p>Number</p></div></div></td><td><div><div><p>Port in which the proxy requests are sent.</p></div></div></td><td><div></div></td><td><div><div><p>x</p></div></div></td></tr><tr><td><p><strong>Username</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Username to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Password</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>Password to authenticate against the proxy.</p></div></div></td><td><div></div></td><td><div></div></td></tr><tr><td><p><strong>Non Proxy Hosts</strong></p></td><td><div><div><p>String</p></div></div></td><td><div><div><p>List of comma separated hosts against which the proxy should not be used.</p></div></div></td><td><div></div></td><td><div></div></td></tr></tbody></table>

## See Also

- [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/latest/)

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=053b520f518790cd38166470ec90787e841ddaeb6e525d6cad7010041f90d2ef url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/mulesoft-inference-connector-xml-maven.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# MuleSoft Inference Connector 1.2.0 XML and Maven Support

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Although you can manually code a Mule app in XML, it is more efficient to use Anypoint Code Builder
or Studio:

- If you manually code a Mule runtime engine (Mule) app in XML, either from the Anypoint Code
  Builder or Studio XML editor, or from a text editor, you can access the connector from your app by
  adding reference statements to both your XML Mule flow and the Apache Maven `pom.xml` file.

- If you add the connector in Anypoint Code Builder or Studio, the XML is automatically populated
  the code with the connector’s namespace and schema location and it also adds a dependency to the
  `pom.xml` file.

## Add a Namespace for the Connector

Paste the following code inside the `mule` tag of the header of your configuration XML:

```xml
http://www.mulesoft.org/schema/mule/mulesoft-inference
http://www.mulesoft.org/schema/mule/mulesoft-inference/current/mule-mulesoft-inference.xsd
```

This example shows how the namespace statements are placed in the `<mule>` XML block:

```xml
<mule xmlns:mulesoft-inference="http://www.mulesoft.org/schema/mule/mulesoft-inference"
	xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
	xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns:file="http://www.mulesoft.org/schema/mule/file"
	xmlns="http://www.mulesoft.org/schema/mule/core"
	xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core
	http://www.mulesoft.org/schema/mule/core/current/mule.xsd
	http://www.mulesoft.org/schema/mule/file
	http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
	http://www.mulesoft.org/schema/mule/http
	http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
	http://www.mulesoft.org/schema/mule/ee/core
	http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd
	http://www.mulesoft.org/schema/mule/mulesoft-inference
	http://www.mulesoft.org/schema/mule/mulesoft-inference/current/mule-mulesoft-inference.xsd">
```

## Add a POM File Dependency

The Apache Maven `pom.xml` file generated by Anypoint Studio contains dependencies for a Mule app.
If you code a Mule app manually, include this XML snippet in your `pom.xml` file to enable access to
this connector:

```xml
<dependency>
  <groupId>com.mulesoft.connectors</groupId>
  <artifactId>mule4-inference-connector</artifactId>
  <version>x.x.x</version>
  <classifier>mule-plugin</classifier>
</dependency>
```

Replace `x.x.x` with the version that corresponds to the connector you are using.

To obtain the most up-to-date `pom.xml` file information:

1.  Go to [Anypoint Exchange](https://www.mulesoft.com/exchange/).

2.  In Exchange, click **Login** and supply your Anypoint Platform username and password.

3.  In Exchange, search for `inference`.

4.  Select the connector.

5.  Click **Dependency Snippets** near the upper right of the screen.

## Next Step

After completing your namespace and pom.xml file, you can try the
[Examples](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-examples).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=1fd4fde01542617b2378eea4d3d5f7b1694f0b281ce5953a2651d60b2d9e12bf url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/mulesoft-inference-connector-examples.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# MuleSoft Inference Connector Examples

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

These examples show you how to use MuleSoft Inference Connector with the OpenAI inference type. To
use other inference types, you must change these configurations.

## Example: Text Generation

This Mule app showcases the use of three distinct text generation operations to interact with
OpenAI. Each operation is demonstrated in its own dedicated flow.

- **\[Agent\] Define Prompt Template**

  This operation is used to create structured prompts for AI agents. In this example, it’s
  configured to enable an agent to analyze customer feedback and determine appropriate actions based
  on sentiment, leveraging the configured OpenAI LLM.

- **\[Chat\] Answer Prompt**

  This operation provides simple, direct question-answering capabilities. The example demonstrates
  its use by asking the LLM a straightforward factual question: `What is the capital of Germany?`.

- **\[Chat\] Completions**

  This operation facilitates multi-turn, context-aware conversations. The example illustrates this
  by providing the LLM with an initial conversational context and then asking it to continue the
  dialogue by asking `What is the capital of Switzerland?`.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:text-generation-config name="OpenAI_Text_generation_config" doc:name="MuleSoft Inference Text generation config" doc:id="3c24c806-bb9f-4512-bc9b-a51cda5c710a" >
		<ms-inference:openai-connection openAIModelName="${config.llmModel}" apiKey="${config.apiKey}" timeout="60000" >
		</ms-inference:openai-connection>
	</ms-inference:text-generation-config>

	<flow name="AgentDefinePromptTemplateFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/prompttemplate"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			{
			"template": "You are a customer satisfaction agent, who analyses the customer feedback in the dataset. Answer via json output and add a type for the result only with positive or negative as well as the complete answer",
			"instructions":"If the customer feedback in the dataset is negative, open a service satisfaction case and apologize to the customer. If the customer feedback in the dataset is positive, thank the customer and wish them a nice day. Do not repeat the feedback and be more direct starting the conversation with formal greetings",
			"dataset": "The training last week was amazing, we learned so much and the trainer was very friendly"
		}]' doc:name="Set Variable" doc:id="209a7846-c3e3-4a8a-9c63-6ab02a772ec4" variableName="testPayload"/>
		<ms-inference:agent-define-prompt-template doc:name="[Agent] Define Prompt Template" doc:id="699b48f4-0ac8-46b5-9f3b-418070e7157d" config-ref="OpenAI_Text_generation_config">
			<ms-inference:template ><![CDATA[#[vars.testPayload.template]]]></ms-inference:template>
			<ms-inference:instructions ><![CDATA[#[vars.testPayload.instructions]]]></ms-inference:instructions>
			<ms-inference:data ><![CDATA[#[vars.testPayload.dataset]]]></ms-inference:data>
		</ms-inference:agent-define-prompt-template>
	</flow>

	<flow name="ChatAnswerPromptFlow" doc:id="698ff181-7968-4294-8f08-51387fbf1dfa" >
		<http:listener doc:name="Listener" doc:id="3ae26012-7b44-41ee-8d6d-bdb51ed9e39e" config-ref="HTTP_Listener_config" path="/chatprompt"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			{
			    "prompt": "What is the capital of Germany?"
			}]' doc:name="Set Variable" doc:id="774dde86-5241-4151-b7fa-b5d7bf412508" variableName="testPayload"/>
		<ms-inference:chat-answer-prompt doc:name="[CHAT] Answer Prompt" doc:id="53e77f54-95c1-42da-b447-b5703212e9ca" config-ref="OpenAI_Text_generation_config">
			<ms-inference:prompt ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:prompt>
		</ms-inference:chat-answer-prompt>
	</flow>

	<flow name="ChatCompletionsFlow" doc:id="ae0b651f-950d-40d0-8c98-9ef050779c06" >
		<http:listener doc:name="Listener" doc:id="a2a68fe6-47da-41b0-954f-3cf51e80d6f2" config-ref="HTTP_Listener_config" path="/chatcompletion"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			[{
			  "role": "assistant",
			  "content": "You are a helpful assistant."
			},
			{
			  "role": "user",
			  "content": "What is the capital of Switzerland!"
			}
			]]' doc:name="Set Variable" doc:id="860b37bd-9fa9-4584-90d4-2723b2e00cc5" variableName="testPayload"/>
		<ms-inference:chat-completions doc:name="[Chat] Completions" doc:id="f2eedc4f-a87b-4579-92a6-4022db9f3c03" config-ref="OpenAI_Text_generation_config">
			<ms-inference:messages ><![CDATA[#[vars.testPayload]]]></ms-inference:messages>
		</ms-inference:chat-completions>
	</flow>
</mule>
```

## Example: Image Generation

This Mule app showcases the use of the **\[Image\] Generate** operation to interact with OpenAI.
This operation is used to generate images based on a prompt. In this example, it’s configured to
generate an image of a penguin dancing on the beach being happy with ice cream.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:image-generation-config name="OpenAIImageGenConfig" doc:name="MuleSoft Inference Image generation config" doc:id="813d08c8-7e6d-4c9e-8259-b80a84093016" >
		<ms-inference:openai-image-connection apiKey="${openai.apiKey}" openAIModelName="${openai.imageGenModel}" />
	</ms-inference:image-generation-config>

	<flow name="GenerateImageFlow" doc:id="698ff181-7968-4294-8f08-51387fbf1dfa" >
		<http:listener doc:name="Listener" doc:id="3ae26012-7b44-41ee-8d6d-bdb51ed9e39e" config-ref="HTTP_Listener_config" path="/generateimage"/>
		<set-variable value='#[%dw 2.0
		   output application/json
		    ---
			{
				"prompt" : "Generate a picture of a penguin dancing on the beach being happy with an ice cream."
			}]' doc:name="Set Variable" doc:id="55ffb06c-d91b-4419-8517-a2d83802413a" variableName="testPayload"/>
		<ms-inference:generate-image doc:name="[Image] Generate (only Base64)" doc:id="9c849d82-b443-4253-bdbd-1fad73df7a45" config-ref="OpenAIImageGenConfig">
			<ms-inference:prompt ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:prompt>
		</ms-inference:generate-image>
	</flow>
</mule>
```

## Example: Toxicity Detection

This Mule app showcases the use of the **\[Toxicity\] Detection by Text** operation to interact with
OpenAI. This operation is used to detect harmful content in a text. In this example, it’s configured
to detect harmful content in the text `You are fat`.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:moderation-config name="OpenAIInvalidApiKeyModerationConfig" doc:name="MuleSoft Inference Moderation config" doc:id="c28800c5-75d8-4904-8e98-a6b8d965e054" >
		<ms-inference:openai-moderation-connection apiKey="${openai.apiKey}" openAIModelName="${openai.llmModel}"/>
	</ms-inference:moderation-config>

	<flow name="ToxicityDetectionFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/toxicitydetection"/>
		<set-variable value='#[%dw 2.0
		   output application/json
		    ---
			{
    			"prompt": "You are fat"
   			}]' doc:name="Set Variable" doc:id="7ced3e9a-a3ec-4768-85b1-081849fca0fc" variableName="testPayload"/>
		<ms-inference:toxicity-detection-text doc:name="[Toxicity] Detection by Text" doc:id="cd45c2f9-cb7f-4fe0-8311-2a58cc6c11ac" config-ref="OpenAIInvalidApiKeyModerationConfig">
			<ms-inference:text ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:text>
		</ms-inference:toxicity-detection-text>
	</flow>
</mule>
```

## Example: Tools

This Mule app showcases the use of the **\[Tools\] Native Template (Reasoning only)** operation to
interact with OpenAI. This operation is used to create autonomous agents that can use external tools
whenever a prompt can’t be answered directly by the AI model. This operation provides a request only
to execute the tools provided in the payload. It doesn’t execute them. In this example, it’s
configured to use the tools to get the current temperature in Zurich in Celsius.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:text-generation-config name="OpenAI_Text_generation_config" doc:name="MuleSoft Inference Text generation config" doc:id="3c24c806-bb9f-4512-bc9b-a51cda5c710a" >
		<ms-inference:openai-connection openAIModelName="${config.llmModel}" apiKey="${config.apiKey}" timeout="60000" >
		</ms-inference:openai-connection>
	</ms-inference:text-generation-config>

	<flow name="ToolsNativeTemplateFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/toolstemplate"/>
		<set-variable
				value="#[%dw 2.0
            output application/json
            ---
            {
                'template': 'You are an helpful assistant',
                'instructions': 'Answer the request with politeness.',
                'dataset': 'What is the current temperature in Zurich in Celsius?',
                'tools': [
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_current_temperature',
                            'description': 'Get the current temperature for a specific location',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'location': {
                                        'type': 'string',
                                        'description': 'The city and state, e.g., San Francisco, CA'
                                    },
                                    'unit': {
                                        'type': 'string',
                                        'enum': ['Celsius', 'Fahrenheit'],
                                        'description': 'The temperature unit to use. Infer this from the user\'s location.'
                                    }
                                },
                                'required': ['location', 'unit']
                            }
                        }
                    },
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_rain_probability',
                            'description': 'Get the probability of rain for a specific location',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'location': {
                                        'type': 'string',
                                        'description': 'The city and state, e.g., San Francisco, CA'
                                    }
                                },
                                'required': ['location']
                            }
                        }
                    },
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_delivery_date',
                            'description': 'Get the delivery date for a customer\'s order. Call this whenever you need to know the delivery date, for example when a customer asks \'Where is my package\'',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'order_id': {
                                        'type': 'string',
                                        'description': 'The customer\'s order ID.'
                                    }
                                },
                                'required': ['order_id']
                            }
                        }
                    }
                ]
            }]"
				doc:name="Set Variable"
				doc:id="9b80bd71-922a-4739-804b-29ac2ba7ef7a"
				variableName="testPayload" />
		<ms-inference:tools-native-template
				doc:name="[Tools] Native Template (Reasoning only)"
				doc:id="e87a2e5e-b19b-4c2d-a4a0-8e991291de97"
				config-ref="OpenAI_Text_generation_config">
			<ms-inference:template><![CDATA[#[vars.testPayload.template]]]></ms-inference:template>
			<ms-inference:instructions><![CDATA[#[vars.testPayload.instructions]]]></ms-inference:instructions>
			<ms-inference:data><![CDATA[#[vars.testPayload.dataset]]]></ms-inference:data>
			<ms-inference:tools><![CDATA[#[vars.testPayload.tools]]]></ms-inference:tools>
		</ms-inference:tools-native-template>
	</flow>
</mule>
```

## See Also

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-additional-config)

- [Using Anypoint Studio to Configure MuleSoft Inference Connector](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-studio)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

<!-- document id=ddc29521f1962c360f035da05c3507394c919354f0360e2a8869ab093d117e2a url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/mulesoft-inference-connector-examples.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# MuleSoft Inference Connector Examples

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

These examples show you how to use MuleSoft Inference Connector with the OpenAI inference type. To
use other inference types, you must change these configurations.

## Example: Text Generation

This Mule app showcases the use of three distinct text generation operations to interact with
OpenAI. Each operation is demonstrated in its own dedicated flow.

- **\[Agent\] Define Prompt Template**

  This operation is used to create structured prompts for AI agents. In this example, it’s
  configured to enable an agent to analyze customer feedback and determine appropriate actions based
  on sentiment, leveraging the configured OpenAI LLM.

- **\[Chat\] Answer Prompt**

  This operation provides simple, direct question-answering capabilities. The example demonstrates
  its use by asking the LLM a straightforward factual question: `What is the capital of Germany?`.

- **\[Chat\] Completions**

  This operation facilitates multi-turn, context-aware conversations. The example illustrates this
  by providing the LLM with an initial conversational context and then asking it to continue the
  dialogue by asking `What is the capital of Switzerland?`.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:text-generation-config name="OpenAI_Text_generation_config" doc:name="MuleSoft Inference Text generation config" doc:id="3c24c806-bb9f-4512-bc9b-a51cda5c710a" >
		<ms-inference:openai-connection openAIModelName="${config.llmModel}" apiKey="${config.apiKey}" timeout="60000" >
		</ms-inference:openai-connection>
	</ms-inference:text-generation-config>

	<flow name="AgentDefinePromptTemplateFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/prompttemplate"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			{
			"template": "You are a customer satisfaction agent, who analyses the customer feedback in the dataset. Answer via json output and add a type for the result only with positive or negative as well as the complete answer",
			"instructions":"If the customer feedback in the dataset is negative, open a service satisfaction case and apologize to the customer. If the customer feedback in the dataset is positive, thank the customer and wish them a nice day. Do not repeat the feedback and be more direct starting the conversation with formal greetings",
			"dataset": "The training last week was amazing, we learned so much and the trainer was very friendly"
		}]' doc:name="Set Variable" doc:id="209a7846-c3e3-4a8a-9c63-6ab02a772ec4" variableName="testPayload"/>
		<ms-inference:agent-define-prompt-template doc:name="[Agent] Define Prompt Template" doc:id="699b48f4-0ac8-46b5-9f3b-418070e7157d" config-ref="OpenAI_Text_generation_config">
			<ms-inference:template ><![CDATA[#[vars.testPayload.template]]]></ms-inference:template>
			<ms-inference:instructions ><![CDATA[#[vars.testPayload.instructions]]]></ms-inference:instructions>
			<ms-inference:data ><![CDATA[#[vars.testPayload.dataset]]]></ms-inference:data>
		</ms-inference:agent-define-prompt-template>
	</flow>

	<flow name="ChatAnswerPromptFlow" doc:id="698ff181-7968-4294-8f08-51387fbf1dfa" >
		<http:listener doc:name="Listener" doc:id="3ae26012-7b44-41ee-8d6d-bdb51ed9e39e" config-ref="HTTP_Listener_config" path="/chatprompt"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			{
			    "prompt": "What is the capital of Germany?"
			}]' doc:name="Set Variable" doc:id="774dde86-5241-4151-b7fa-b5d7bf412508" variableName="testPayload"/>
		<ms-inference:chat-answer-prompt doc:name="[CHAT] Answer Prompt" doc:id="53e77f54-95c1-42da-b447-b5703212e9ca" config-ref="OpenAI_Text_generation_config">
			<ms-inference:prompt ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:prompt>
		</ms-inference:chat-answer-prompt>
	</flow>

	<flow name="ChatCompletionsFlow" doc:id="ae0b651f-950d-40d0-8c98-9ef050779c06" >
		<http:listener doc:name="Listener" doc:id="a2a68fe6-47da-41b0-954f-3cf51e80d6f2" config-ref="HTTP_Listener_config" path="/chatcompletion"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			[{
			  "role": "assistant",
			  "content": "You are a helpful assistant."
			},
			{
			  "role": "user",
			  "content": "What is the capital of Switzerland!"
			}
			]]' doc:name="Set Variable" doc:id="860b37bd-9fa9-4584-90d4-2723b2e00cc5" variableName="testPayload"/>
		<ms-inference:chat-completions doc:name="[Chat] Completions" doc:id="f2eedc4f-a87b-4579-92a6-4022db9f3c03" config-ref="OpenAI_Text_generation_config">
			<ms-inference:messages ><![CDATA[#[vars.testPayload]]]></ms-inference:messages>
		</ms-inference:chat-completions>
	</flow>
</mule>
```

## Example: Image Generation

This Mule app showcases the use of the **\[Image\] Generate** operation to interact with OpenAI.
This operation is used to generate images based on a prompt. In this example, it’s configured to
generate an image of a penguin dancing on the beach being happy with ice cream.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:image-generation-config name="OpenAIImageGenConfig" doc:name="MuleSoft Inference Image generation config" doc:id="813d08c8-7e6d-4c9e-8259-b80a84093016" >
		<ms-inference:openai-image-connection apiKey="${openai.apiKey}" openAIModelName="${openai.imageGenModel}" />
	</ms-inference:image-generation-config>

	<flow name="GenerateImageFlow" doc:id="698ff181-7968-4294-8f08-51387fbf1dfa" >
		<http:listener doc:name="Listener" doc:id="3ae26012-7b44-41ee-8d6d-bdb51ed9e39e" config-ref="HTTP_Listener_config" path="/generateimage"/>
		<set-variable value='#[%dw 2.0
		   output application/json
		    ---
			{
				"prompt" : "Generate a picture of a penguin dancing on the beach being happy with an ice cream."
			}]' doc:name="Set Variable" doc:id="55ffb06c-d91b-4419-8517-a2d83802413a" variableName="testPayload"/>
		<ms-inference:generate-image doc:name="[Image] Generate (only Base64)" doc:id="9c849d82-b443-4253-bdbd-1fad73df7a45" config-ref="OpenAIImageGenConfig">
			<ms-inference:prompt ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:prompt>
		</ms-inference:generate-image>
	</flow>
</mule>
```

## Example: Toxicity Detection

This Mule app showcases the use of the **\[Toxicity\] Detection by Text** operation to interact with
OpenAI. This operation is used to detect harmful content in a text. In this example, it’s configured
to detect harmful content in the text `You are fat`.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:moderation-config name="OpenAIInvalidApiKeyModerationConfig" doc:name="MuleSoft Inference Moderation config" doc:id="c28800c5-75d8-4904-8e98-a6b8d965e054" >
		<ms-inference:openai-moderation-connection apiKey="${openai.apiKey}" openAIModelName="${openai.llmModel}"/>
	</ms-inference:moderation-config>

	<flow name="ToxicityDetectionFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/toxicitydetection"/>
		<set-variable value='#[%dw 2.0
		   output application/json
		    ---
			{
    			"prompt": "You are fat"
   			}]' doc:name="Set Variable" doc:id="7ced3e9a-a3ec-4768-85b1-081849fca0fc" variableName="testPayload"/>
		<ms-inference:toxicity-detection-text doc:name="[Toxicity] Detection by Text" doc:id="cd45c2f9-cb7f-4fe0-8311-2a58cc6c11ac" config-ref="OpenAIInvalidApiKeyModerationConfig">
			<ms-inference:text ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:text>
		</ms-inference:toxicity-detection-text>
	</flow>
</mule>
```

## Example: Tools

This Mule app showcases the use of the **\[Tools\] Native Template (Reasoning only)** operation to
interact with OpenAI. This operation is used to create autonomous agents that can use external tools
whenever a prompt can’t be answered directly by the AI model. This operation provides a request only
to execute the tools provided in the payload. It doesn’t execute them. In this example, it’s
configured to use the tools to get the current temperature in Zurich in Celsius.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:text-generation-config name="OpenAI_Text_generation_config" doc:name="MuleSoft Inference Text generation config" doc:id="3c24c806-bb9f-4512-bc9b-a51cda5c710a" >
		<ms-inference:openai-connection openAIModelName="${config.llmModel}" apiKey="${config.apiKey}" timeout="60000" >
		</ms-inference:openai-connection>
	</ms-inference:text-generation-config>

	<flow name="ToolsNativeTemplateFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/toolstemplate"/>
		<set-variable
				value="#[%dw 2.0
            output application/json
            ---
            {
                'template': 'You are an helpful assistant',
                'instructions': 'Answer the request with politeness.',
                'dataset': 'What is the current temperature in Zurich in Celsius?',
                'tools': [
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_current_temperature',
                            'description': 'Get the current temperature for a specific location',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'location': {
                                        'type': 'string',
                                        'description': 'The city and state, e.g., San Francisco, CA'
                                    },
                                    'unit': {
                                        'type': 'string',
                                        'enum': ['Celsius', 'Fahrenheit'],
                                        'description': 'The temperature unit to use. Infer this from the user\'s location.'
                                    }
                                },
                                'required': ['location', 'unit']
                            }
                        }
                    },
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_rain_probability',
                            'description': 'Get the probability of rain for a specific location',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'location': {
                                        'type': 'string',
                                        'description': 'The city and state, e.g., San Francisco, CA'
                                    }
                                },
                                'required': ['location']
                            }
                        }
                    },
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_delivery_date',
                            'description': 'Get the delivery date for a customer\'s order. Call this whenever you need to know the delivery date, for example when a customer asks \'Where is my package\'',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'order_id': {
                                        'type': 'string',
                                        'description': 'The customer\'s order ID.'
                                    }
                                },
                                'required': ['order_id']
                            }
                        }
                    }
                ]
            }]"
				doc:name="Set Variable"
				doc:id="9b80bd71-922a-4739-804b-29ac2ba7ef7a"
				variableName="testPayload" />
		<ms-inference:tools-native-template
				doc:name="[Tools] Native Template (Reasoning only)"
				doc:id="e87a2e5e-b19b-4c2d-a4a0-8e991291de97"
				config-ref="OpenAI_Text_generation_config">
			<ms-inference:template><![CDATA[#[vars.testPayload.template]]]></ms-inference:template>
			<ms-inference:instructions><![CDATA[#[vars.testPayload.instructions]]]></ms-inference:instructions>
			<ms-inference:data><![CDATA[#[vars.testPayload.dataset]]]></ms-inference:data>
			<ms-inference:tools><![CDATA[#[vars.testPayload.tools]]]></ms-inference:tools>
		</ms-inference:tools-native-template>
	</flow>
</mule>
```

## See Also

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-additional-config)

- [Using Anypoint Studio to Configure MuleSoft Inference Connector](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-studio)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

<!-- document id=430a314aeeb473480e1503575f2501428000475ba5e27ba1bc41b4c7b4f2c4bf url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/mulesoft-inference-connector-examples.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# MuleSoft Inference Connector Examples

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

These examples show you how to use MuleSoft Inference Connector with the OpenAI inference type. To
use other inference types, you must change these configurations.

## Example: Text Generation

This Mule app showcases the use of three distinct text generation operations to interact with
OpenAI. Each operation is demonstrated in its own dedicated flow.

- **\[Agent\] Define Prompt Template**

  This operation is used to create structured prompts for AI agents. In this example, it’s
  configured to enable an agent to analyze customer feedback and determine appropriate actions based
  on sentiment, leveraging the configured OpenAI LLM.

- **\[Chat\] Answer Prompt**

  This operation provides simple, direct question-answering capabilities. The example demonstrates
  its use by asking the LLM a straightforward factual question: `What is the capital of Germany?`.

- **\[Chat\] Completions**

  This operation facilitates multi-turn, context-aware conversations. The example illustrates this
  by providing the LLM with an initial conversational context and then asking it to continue the
  dialogue by asking `What is the capital of Switzerland?`.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:text-generation-config name="OpenAI_Text_generation_config" doc:name="MuleSoft Inference Text generation config" doc:id="3c24c806-bb9f-4512-bc9b-a51cda5c710a" >
		<ms-inference:openai-connection openAIModelName="${config.llmModel}" apiKey="${config.apiKey}" timeout="60000" >
		</ms-inference:openai-connection>
	</ms-inference:text-generation-config>

	<flow name="AgentDefinePromptTemplateFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/prompttemplate"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			{
			"template": "You are a customer satisfaction agent, who analyses the customer feedback in the dataset. Answer via json output and add a type for the result only with positive or negative as well as the complete answer",
			"instructions":"If the customer feedback in the dataset is negative, open a service satisfaction case and apologize to the customer. If the customer feedback in the dataset is positive, thank the customer and wish them a nice day. Do not repeat the feedback and be more direct starting the conversation with formal greetings",
			"dataset": "The training last week was amazing, we learned so much and the trainer was very friendly"
		}]' doc:name="Set Variable" doc:id="209a7846-c3e3-4a8a-9c63-6ab02a772ec4" variableName="testPayload"/>
		<ms-inference:agent-define-prompt-template doc:name="[Agent] Define Prompt Template" doc:id="699b48f4-0ac8-46b5-9f3b-418070e7157d" config-ref="OpenAI_Text_generation_config">
			<ms-inference:template ><![CDATA[#[vars.testPayload.template]]]></ms-inference:template>
			<ms-inference:instructions ><![CDATA[#[vars.testPayload.instructions]]]></ms-inference:instructions>
			<ms-inference:data ><![CDATA[#[vars.testPayload.dataset]]]></ms-inference:data>
		</ms-inference:agent-define-prompt-template>
	</flow>

	<flow name="ChatAnswerPromptFlow" doc:id="698ff181-7968-4294-8f08-51387fbf1dfa" >
		<http:listener doc:name="Listener" doc:id="3ae26012-7b44-41ee-8d6d-bdb51ed9e39e" config-ref="HTTP_Listener_config" path="/chatprompt"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			{
			    "prompt": "What is the capital of Germany?"
			}]' doc:name="Set Variable" doc:id="774dde86-5241-4151-b7fa-b5d7bf412508" variableName="testPayload"/>
		<ms-inference:chat-answer-prompt doc:name="[CHAT] Answer Prompt" doc:id="53e77f54-95c1-42da-b447-b5703212e9ca" config-ref="OpenAI_Text_generation_config">
			<ms-inference:prompt ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:prompt>
		</ms-inference:chat-answer-prompt>
	</flow>

	<flow name="ChatCompletionsFlow" doc:id="ae0b651f-950d-40d0-8c98-9ef050779c06" >
		<http:listener doc:name="Listener" doc:id="a2a68fe6-47da-41b0-954f-3cf51e80d6f2" config-ref="HTTP_Listener_config" path="/chatcompletion"/>
		<set-variable value='#[%dw 2.0
			output application/json
			---
			[{
			  "role": "assistant",
			  "content": "You are a helpful assistant."
			},
			{
			  "role": "user",
			  "content": "What is the capital of Switzerland!"
			}
			]]' doc:name="Set Variable" doc:id="860b37bd-9fa9-4584-90d4-2723b2e00cc5" variableName="testPayload"/>
		<ms-inference:chat-completions doc:name="[Chat] Completions" doc:id="f2eedc4f-a87b-4579-92a6-4022db9f3c03" config-ref="OpenAI_Text_generation_config">
			<ms-inference:messages ><![CDATA[#[vars.testPayload]]]></ms-inference:messages>
		</ms-inference:chat-completions>
	</flow>
</mule>
```

## Example: Image Generation

This Mule app showcases the use of the **\[Image\] Generate** operation to interact with OpenAI.
This operation is used to generate images based on a prompt. In this example, it’s configured to
generate an image of a penguin dancing on the beach being happy with ice cream.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:image-generation-config name="OpenAIImageGenConfig" doc:name="MuleSoft Inference Image generation config" doc:id="813d08c8-7e6d-4c9e-8259-b80a84093016" >
		<ms-inference:openai-image-connection apiKey="${openai.apiKey}" openAIModelName="${openai.imageGenModel}" />
	</ms-inference:image-generation-config>

	<flow name="GenerateImageFlow" doc:id="698ff181-7968-4294-8f08-51387fbf1dfa" >
		<http:listener doc:name="Listener" doc:id="3ae26012-7b44-41ee-8d6d-bdb51ed9e39e" config-ref="HTTP_Listener_config" path="/generateimage"/>
		<set-variable value='#[%dw 2.0
		   output application/json
		    ---
			{
				"prompt" : "Generate a picture of a penguin dancing on the beach being happy with an ice cream."
			}]' doc:name="Set Variable" doc:id="55ffb06c-d91b-4419-8517-a2d83802413a" variableName="testPayload"/>
		<ms-inference:generate-image doc:name="[Image] Generate (only Base64)" doc:id="9c849d82-b443-4253-bdbd-1fad73df7a45" config-ref="OpenAIImageGenConfig">
			<ms-inference:prompt ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:prompt>
		</ms-inference:generate-image>
	</flow>
</mule>
```

## Example: Toxicity Detection

This Mule app showcases the use of the **\[Toxicity\] Detection by Text** operation to interact with
OpenAI. This operation is used to detect harmful content in a text. In this example, it’s configured
to detect harmful content in the text `You are fat`.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:moderation-config name="OpenAIInvalidApiKeyModerationConfig" doc:name="MuleSoft Inference Moderation config" doc:id="c28800c5-75d8-4904-8e98-a6b8d965e054" >
		<ms-inference:openai-moderation-connection apiKey="${openai.apiKey}" openAIModelName="${openai.llmModel}"/>
	</ms-inference:moderation-config>

	<flow name="ToxicityDetectionFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/toxicitydetection"/>
		<set-variable value='#[%dw 2.0
		   output application/json
		    ---
			{
    			"prompt": "You are fat"
   			}]' doc:name="Set Variable" doc:id="7ced3e9a-a3ec-4768-85b1-081849fca0fc" variableName="testPayload"/>
		<ms-inference:toxicity-detection-text doc:name="[Toxicity] Detection by Text" doc:id="cd45c2f9-cb7f-4fe0-8311-2a58cc6c11ac" config-ref="OpenAIInvalidApiKeyModerationConfig">
			<ms-inference:text ><![CDATA[#[vars.testPayload.prompt]]]></ms-inference:text>
		</ms-inference:toxicity-detection-text>
	</flow>
</mule>
```

## Example: Tools

This Mule app showcases the use of the **\[Tools\] Native Template (Reasoning only)** operation to
interact with OpenAI. This operation is used to create autonomous agents that can use external tools
whenever a prompt can’t be answered directly by the AI model. This operation provides a request only
to execute the tools provided in the payload. It doesn’t execute them. In this example, it’s
configured to use the tools to get the current temperature in Zurich in Celsius.

```xml
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ms-inference="http://www.mulesoft.org/schema/mule/ms-inference" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	  xmlns="http://www.mulesoft.org/schema/mule/core"
	  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ms-inference http://www.mulesoft.org/schema/mule/ms-inference/current/mule-ms-inference.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="1d867172-d19a-4ab1-b194-52c2ad455d7f" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>

	<ms-inference:text-generation-config name="OpenAI_Text_generation_config" doc:name="MuleSoft Inference Text generation config" doc:id="3c24c806-bb9f-4512-bc9b-a51cda5c710a" >
		<ms-inference:openai-connection openAIModelName="${config.llmModel}" apiKey="${config.apiKey}" timeout="60000" >
		</ms-inference:openai-connection>
	</ms-inference:text-generation-config>

	<flow name="ToolsNativeTemplateFlow" doc:id="1ee32ee4-8c15-4636-9a49-0784f782cb98" >
		<http:listener doc:name="Listener" doc:id="8725c446-5f45-4732-8fca-ea9006aabed6" config-ref="HTTP_Listener_config" path="/toolstemplate"/>
		<set-variable
				value="#[%dw 2.0
            output application/json
            ---
            {
                'template': 'You are an helpful assistant',
                'instructions': 'Answer the request with politeness.',
                'dataset': 'What is the current temperature in Zurich in Celsius?',
                'tools': [
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_current_temperature',
                            'description': 'Get the current temperature for a specific location',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'location': {
                                        'type': 'string',
                                        'description': 'The city and state, e.g., San Francisco, CA'
                                    },
                                    'unit': {
                                        'type': 'string',
                                        'enum': ['Celsius', 'Fahrenheit'],
                                        'description': 'The temperature unit to use. Infer this from the user\'s location.'
                                    }
                                },
                                'required': ['location', 'unit']
                            }
                        }
                    },
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_rain_probability',
                            'description': 'Get the probability of rain for a specific location',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'location': {
                                        'type': 'string',
                                        'description': 'The city and state, e.g., San Francisco, CA'
                                    }
                                },
                                'required': ['location']
                            }
                        }
                    },
                    {
                        'type': 'function',
                        'function': {
                            'name': 'get_delivery_date',
                            'description': 'Get the delivery date for a customer\'s order. Call this whenever you need to know the delivery date, for example when a customer asks \'Where is my package\'',
                            'parameters': {
                                'type': 'object',
                                'properties': {
                                    'order_id': {
                                        'type': 'string',
                                        'description': 'The customer\'s order ID.'
                                    }
                                },
                                'required': ['order_id']
                            }
                        }
                    }
                ]
            }]"
				doc:name="Set Variable"
				doc:id="9b80bd71-922a-4739-804b-29ac2ba7ef7a"
				variableName="testPayload" />
		<ms-inference:tools-native-template
				doc:name="[Tools] Native Template (Reasoning only)"
				doc:id="e87a2e5e-b19b-4c2d-a4a0-8e991291de97"
				config-ref="OpenAI_Text_generation_config">
			<ms-inference:template><![CDATA[#[vars.testPayload.template]]]></ms-inference:template>
			<ms-inference:instructions><![CDATA[#[vars.testPayload.instructions]]]></ms-inference:instructions>
			<ms-inference:data><![CDATA[#[vars.testPayload.dataset]]]></ms-inference:data>
			<ms-inference:tools><![CDATA[#[vars.testPayload.tools]]]></ms-inference:tools>
		</ms-inference:tools-native-template>
	</flow>
</mule>
```

## Example: MCP Tools

This Mule application showcases the use of the **\[MCP\] Tooling** operation to interact with OpenAI
via MCP Connector. This operation enables you to execute tools hosted on an MCP server directly
through user-defined instructions. In this example, the operation uses the tools to check the
inventory status for a specific part number, `MULETEST0`.

```xml
<flow name="TestNewInferenceFlow2">
	<http:listener path="path" doc:name="Listener" doc:id="zecmdx" config-ref="HTTP_Listener_config" />
	<set-variable doc:id="7edeecec-4d47-4bfe-a699-23987b60ac14" doc:name="Set Variable" value="#[%dw 2.0
			output application/java
			---
			{
				'template': 'You are an helpful assistant',
				'instructions':'Answer the request with politeness.',
				'dataset': 'Check Inventory for MULETEST0'

			}]" variableName="testPayload" />
	<ms-inference:mcp-tools-native-template doc:name="[MCP] Tooling" doc:id="6783c672-bc72-49c7-bf3a-3f3379e3c9a9" config-ref="Text-generation-config">
		<ms-inference:mcp-config-references>
			<ms-inference:mcp-config mcpClientConfigRef="MCP_Client_SSE_ERP" />
			<ms-inference:mcp-config mcpClientConfigRef="MCP_Client_HTTP" />
		</ms-inference:mcp-config-references>
		<ms-inference:template>
			<![CDATA[#[vars.testPayload.template]]]>
		</ms-inference:template>
		<ms-inference:instructions>
			<![CDATA[#[vars.testPayload.instructions]]]>
		</ms-inference:instructions>
		<ms-inference:data>
			<![CDATA[#[vars.testPayload.dataset]]]>
		</ms-inference:data>
	</ms-inference:mcp-tools-native-template>
	<ee:transform doc:id="8eda33bd-db2f-412a-b448-dc633d326a39" doc:name="Transform Message">
		<ee:message>
			<ee:set-payload>
				<![CDATA[%dw 2.0
                output application/json
                ---
                {
                    payload: payload,
                    attributes: attributes
                }]]>
			</ee:set-payload>
		</ee:message>
	</ee:transform>
</flow>

<!-- MCP client configs -->
<mcp:client-config name="MCP_Client_SSE_ERP" doc:name="MCP Client" doc:id="fd3bb75d-82e1-4aed-a1e3-f7c7ca6e28fb" clientName="MCP_Client_SSE_ERP" clientVersion="1.0.0">
        <mcp:sse-client-connection requestTimeout="60" serverUrl="${mcp.erpServer}" />
 </mcp:client-config>


<mcp:client-config name="MCP_Client_HTTP" doc:name="MCP Client" doc:id="163c2f4f-86bd-4860-b3b1-5606259f324f" clientName="MCP_Client_HTTP" clientVersion="1.0.0">
        <mcp:streamable-http-client-connection serverUrl="${mcp.httpServer}">
            <mcp:default-request-headers>
                <mcp:default-request-header key="client_id" value="${mcp.httpServer.clientId}" />
                <mcp:default-request-header key="client_secret" value="${mcp.httpServer.clientSecret}" />
            </mcp:default-request-headers>
        </mcp:streamable-http-client-connection>
    </mcp:client-config>
```

## Example: Additional Request Attributes

This feature showcases the ability to control the formatting of LLM outputs directly from the
connector using the **Additional Request Attributes** parameter. With this parameter, you can now
specify a JSON object with attributes that are flattened into the root level of the request payload.
In this example, you can directly pass a `response_format` with type `json_schema` followed by the
corresponding JSON schema. This is included in the request sent to the Inference API, enabling
consistent and structured output in the provided JSON schema.

```xml
<ms-inference:chat-answer-prompt doc:name="[Chat] Answer Prompt" doc:id="fe114415-0f76-4f8f-9bd0-968f1cf2c3e5" config-ref="MuleSoft_Inference_Text_generation_confi_OpenAI">
	<ms-inference:prompt ><![CDATA[What is the capital of India? give country and city in a json format]]></ms-inference:prompt>

	<ms-inference:additional-request-attributes ><![CDATA[{
		  "response_format": {
		    "type": "json_schema",
		    "json_schema": {
		      "name": "location_schema",
		      "schema": {
		        "type": "object",
		        "properties": {
		          "countryName": {
		            "description": "The name of the country.",
		            "type": "string"
		          },
		          "city": {
		            "description": "The name of the city.",
		            "type": "string"
		          }
		        },
		        "required": [
		          "countryName",
		          "city"
		        ]
		      }
		    }
		  }
		}]]>
	</ms-inference:additional-request-attributes>

</ms-inference:chat-answer-prompt>
```

## See Also

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-additional-config)

- [Using Anypoint Studio to Configure MuleSoft Inference Connector](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-studio)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

<!-- document id=e581a108ece0df87513208f9dcb5ea0d8ad2f67acd7f2e0b76ca86c16b76086d url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/mulesoft-inference-connector-config-acb.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# Using Anypoint Code Builder to Configure MuleSoft Inference Connector 1.0

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Cloud IDE (Beta) is scheduled for retirement on August 31, 2026. August 30, 2026 is the last day you
can access Cloud IDE (Beta). Download your projects to work from Desktop IDE. Learn more in
[Migrate a Project from Cloud IDE to Desktop IDE](https://docs.mulesoft.com/anypoint-code-builder/int-import-mule-project#migrate-cloud-to-desktop).

To use Anypoint Code Builder to configure a connector or module, create a basic integration project
in Anypoint Code Builder, add connectors and other components to your Mule application to process
your business logic, and configure the attributes.

## Before You Begin

Before creating an integration project, you must:

- [Set up and access the Anypoint Code Builder web or desktop IDE](https://docs.mulesoft.com/anypoint-code-builder/start-acb)

- Have credentials to access the connector’s API

- Ensure that the Visual Studio Code Extension Pack for Java is installed.

To use this connector, you must be familiar with:

- The connector’s API

- Anypoint Connectors

- Mule runtime engine (Mule)

## Create a New Integration Project

To create a new integration project:

1.  In the activity bar of the IDE, click the
    ![Anypoint Code Builder icon](/mulesoft/current/en-us/connectors/_images/acb-icon-ms-logo.png) (**Anypoint Code
    Builder**) icon.

2.  From **Quick Actions**, click **Develop an Integration**:

    ![Develop an Integration link highlighted in the Getting started section](/mulesoft/current/en-us/anypoint-code-builder/_images/develop-integration-option-mat.png)

3.  Complete the following in the **Develop an Integration** form:

    <table><colgroup><col> <col></colgroup><thead><tr><th>Attribute</th><th>Value</th></tr></thead><tbody><tr><td><div><div><p><strong>Project Name</strong></p></div></div></td><td><div><div><p>Unique name for your project.</p></div><div><p>This name is used as the title and name of the integration project file. For example, if the project name is "Salesforce Integration," the project file name is <code>salesforce-integration</code>.</p></div></div></td></tr><tr><td><div><div><p><strong>Project Location</strong></p></div></div></td><td><div><div><p>Your home directory or another directory you create. Click <strong>Browse</strong> to select a different directory for the integration project.</p></div><div><p>Don’t create the project within another project directory.</p></div><div><p>See <a href="https://docs.mulesoft.com/anypoint-code-builder/start-add-folders">Adding Folders to Your Home Directory</a>.</p></div></div></td></tr></tbody></table>

4.  Select the type of project to create by selecting either the **Empty Project** or **Template or
    Example** card.

5.  Select the **Mule runtime** and **Java Version** for your app.

6.  Click **Create Project**.

## Add Components to Your Project

Add components to your integration project to build a flow:

1.  In the **Explorer** view, open the XML file for your project, such as `my-project-name.xml`.

2.  Select **Build a Flow** from the start card to create an empty flow:

    ![Anypoint Code Builder canvas with starting cards with the options to start from scratch, build a flow, or build a subflow](/mulesoft/current/en-us/connectors/_images/acb-build-flow.png)

3.  Change the name of the flow:

    1.  Click **Flow**.

    2.  Click the edit icon.

        ![Arrow pointing to the flow name edit pencil icon](/mulesoft/current/en-us/connectors/_images/acb-change-flow-name.png)

    3.  Enter the flow name, and click the checkmark.

    4.  In the canvas UI, click the
        ![Add component icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-plus.png) (**Add component**)
        icon.

    5.  In the **Add Component** panel, search for and select your component from the results. The
        following example shows the **Listener** component from the **HTTP** search results:

        ![Listener component highlighted in the Add Component section](/mulesoft/current/en-us/anypoint-code-builder/_images/main-tutorial-add-first-listener.png)

        The configuration XML file now includes the XML for the HTTP Listener into the `<flow/>`
        element, for example:

        ```XML
        <flow name="getFlights" >
          <http:listener path="" config-ref="" doc:name="Listener" doc:id="ojzfry" />
        </flow>
        ```

## Configure the Trigger For the Flow

A trigger (source) starts the flow when specified conditions are met. If a connector has its own
associated sources, you can use a connector-specific trigger to initiate the flow. Each connector
trigger has a unique configuration.

You can configure the following sources for this connector:

- **HTTP > Listener**

  Initiates a flow each time it receives a request on the configured host and port

- **Scheduler**

  Initiates a flow when a time-based condition is met

For example, to configure an HTTP Listener trigger, follow these steps:

1.  Click the **Listener** component on the canvas UI.

2.  (Optional) Click the edit icon to change the name of the **Listener** component.

3.  (Optional) If you created a configuration file for the connector, select it.

4.  In the **General** tab, in the **Path** field, enter the relative path for the path set in the
    HTTP configuration element. See
    [Configure Paths](https://docs.mulesoft.com/http-connector/latest/http-listener-ref#configure-paths).

5.  Select the **Advanced** tab to configure the following attributes:

    <table><colgroup><col> <col> <col></colgroup><thead><tr><th>Attribute</th><th>Description</th><th>Required</th></tr></thead><tbody><tr><td><p>Allowed methods</p></td><td><p>To allow all HTTP methods (default), do not define this attribute. To restrict certain methods, specify all allowed HTTP methods, for example, <code>GET, POST</code>, as a comma-separated list.</p></td><td><p>No</p></td></tr><tr><td><p>Response streaming mode</p></td><td><p>Whether to use streaming when the response is sent.</p></td><td><p>No</p></td></tr><tr><td><p>Output MIME type</p></td><td><p>MIME type of the payload that this component outputs.</p></td><td><p>No</p></td></tr><tr><td><p>Parameters</p></td><td><p>Appends a key-value pair for a reader property to the value of the Output MIME attribute. Multiple key-value pairs are allowed.</p></td><td><p>No</p></td></tr><tr><td><p>Output encoding</p></td><td><p>Encoding of the payload that this component outputs.</p></td><td><p>No</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><p>Configures the way data streams are consumed in Mule apps.</p></td><td><p>No</p></td></tr><tr><td><p>Primary node only</p></td><td><p>Select this option to execute the <strong>Listener</strong> trigger on the primary node only when running in a cluster.</p></td><td><p>No</p></td></tr><tr><td><p>Redelivery policy</p></td><td><p>Configures the redelivery policy for executing requests that generate errors. You can add a redelivery policy to any source in a flow.</p></td><td><p>No</p></td></tr><tr><td><p>Reconnection strategy</p></td><td><p>When an operation in a Mule application fails to connect to an external server, the default behavior is for the operation to fail immediately and return a connectivity error. You can modify the default behavior by configuring the reconnection strategy for the operation.</p></td><td><p>No</p></td></tr><tr><td><p>Response</p></td><td><p>See <a href="https://docs.mulesoft.com/http-connector/latest/http-documentation#HttpResponseAttributes">HTTP Response Attributes</a>.</p></td><td><p>No</p></td></tr><tr><td><p>Error Response</p></td><td><p>HTTP error response returns a status code for the error.</p></td><td><p>No</p></td></tr><tr><td><p>Body</p></td><td><p>Body of the error message.</p></td><td><p>No</p></td></tr><tr><td><p>Headers</p></td><td><p>Headers to include in the error message.</p></td><td><p>No</p></td></tr><tr><td><p>Status code</p></td><td><p>Status code of the error response, for example, <code>500</code>.</p></td><td><p>No</p></td></tr><tr><td><p>Reason phrase</p></td><td><p>Reason for the error, for example, <code>Gateway Timeout</code> or <code>Internal Server Error</code>.</p></td><td><p>No</p></td></tr></tbody></table>

## Add the Connector to Your Integration Project

1.  In the Explorer view, open the configuration XML file for your project, such as
    `my-project-name.xml`.

2.  Click the
    ![Show Mule graphical mode icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-tree-view.png) (**Show
    Mule graphical mode**) icon in the activity bar to open the canvas UI if it doesn’t open
    automatically.

3.  Add the connector the same way you added other components from the canvas UI:

    1.  In the canvas UI, click the
        ![Add component icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-plus.png) (**Add component**)
        icon.

    2.  In the **Add Component** panel, click **Connectors**.

4.  Click the
    ![Search in Exchange icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-search-exchange.png)
    (**Search in Exchange**) toggle:

    ![Search in Exchange toggle](/mulesoft/current/en-us/anypoint-code-builder/_images/int-connector-search-exchange.png)

5.  Enter `mulesoft inference` in the search box.

6.  Select the connector to add it to your project.

Adding a connector to an integration project does not make that connector available to other
projects in your Anypoint Code Builder workspace.

## Add a Global Configuration

1.  In the configuration XML, place your cursor before the `<flow/>` element and type
    `ms-inference`.

    Ensure that the cursor is not inside the `<flow/>` element.

2.  Select the snippet. Here are some examples (using OpenAI) based on the available configurations:

    ```xml
    <ms-inference:text-generation-config name="OpenAITextGen" doc:name="MuleSoft Inference Text generation config" doc:id="1ef66d01-d3ec-442a-bf61-99919d73f97e" >
    <ms-inference:openai-connection apiKey="${openai.apiKey}" openAIModelName="${openai.llmModel}" timeout="60000" />
    </ms-inference:text-generation-config>
    ```

    ```xml
    <ms-inference:moderation-config name="OpenAIModeration" doc:name="MuleSoft Inference Moderation config" doc:id="8bd9c1fe-3176-4319-9d91-090e4e6da3cc" >
        <ms-inference:openai-moderation-connection apiKey="${openai.apiKey}" openAIModelName="${openai.moderation}"/>
    </ms-inference:moderation-config>
    ```

    ```xml
    <ms-inference:image-generation-config name="OpenAIImageGen" doc:name="MuleSoft Inference Image generation config" doc:id="aba51427-1dd3-4915-a411-8fe6543ee6ac" >
        <ms-inference:openai-image-connection apiKey="${openai.apiKey}" openAIModelName="${openai.imageGenModel}" />
    </ms-inference:image-generation-config>
    ```

    ```xml
    <ms-inference:vision-config name="OpenAIVision" doc:name="MuleSoft Inference Vision config" doc:id="797661b2-a4ac-493d-8550-45d05d03da17" >
        <ms-inference:openai-vision-connection apiKey="${openai.apiKey}" openAIModelName="${openai.visionModel}" />
    </ms-inference:vision-config>
    ```

3.  Add the attributes to complete the configuration.

## Configure Provider-Specific Settings

Based on your chosen inference provider, you may need to configure additional settings. For example:

```xml
<ms-inference:text-generation-config
    provider="OPENAI"
    modelName="gpt-4o"
    temperature="0.7"
    maxTokens="1000"
    timeout="60"
    name="OpenAIConfig">
</ms-inference:text-generation-config>
```

## Configure Additional Connector Fields

After you configure a global element for the connector, configure the other required fields. The
required fields vary depending on which connector actions you use.

For more information about configuring MuleSoft Inference Connector actions, see
[MuleSoft Inference Connector - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-additional-config).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-additional-config)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=035d1fae7d14a883f42ee664ddf0149ebb5cd3fa880db71c87abc208113dcffe url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/mulesoft-inference-connector-config-acb.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# Using Anypoint Code Builder to Configure MuleSoft Inference Connector 1.1

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Cloud IDE (Beta) is scheduled for retirement on August 31, 2026. August 30, 2026 is the last day you
can access Cloud IDE (Beta). Download your projects to work from Desktop IDE. Learn more in
[Migrate a Project from Cloud IDE to Desktop IDE](https://docs.mulesoft.com/anypoint-code-builder/int-import-mule-project#migrate-cloud-to-desktop).

To use Anypoint Code Builder to configure a connector or module, create a basic integration project
in Anypoint Code Builder, add connectors and other components to your Mule application to process
your business logic, and configure the attributes.

## Before You Begin

Before creating an integration project, you must:

- [Set up and access the Anypoint Code Builder web or desktop IDE](https://docs.mulesoft.com/anypoint-code-builder/start-acb)

- Have credentials to access the connector’s API

- Ensure that the Visual Studio Code Extension Pack for Java is installed.

To use this connector, you must be familiar with:

- The connector’s API

- Anypoint Connectors

- Mule runtime engine (Mule)

## Create a New Integration Project

To create a new integration project:

1.  In the activity bar of the IDE, click the
    ![Anypoint Code Builder icon](/mulesoft/current/en-us/connectors/_images/acb-icon-ms-logo.png) (**Anypoint Code
    Builder**) icon.

2.  From **Quick Actions**, click **Develop an Integration**:

    ![Develop an Integration link highlighted in the Getting started section](/mulesoft/current/en-us/anypoint-code-builder/_images/develop-integration-option-mat.png)

3.  Complete the following in the **Develop an Integration** form:

    <table><colgroup><col> <col></colgroup><thead><tr><th>Attribute</th><th>Value</th></tr></thead><tbody><tr><td><div><div><p><strong>Project Name</strong></p></div></div></td><td><div><div><p>Unique name for your project.</p></div><div><p>This name is used as the title and name of the integration project file. For example, if the project name is "Salesforce Integration," the project file name is <code>salesforce-integration</code>.</p></div></div></td></tr><tr><td><div><div><p><strong>Project Location</strong></p></div></div></td><td><div><div><p>Your home directory or another directory you create. Click <strong>Browse</strong> to select a different directory for the integration project.</p></div><div><p>Don’t create the project within another project directory.</p></div><div><p>See <a href="https://docs.mulesoft.com/anypoint-code-builder/start-add-folders">Adding Folders to Your Home Directory</a>.</p></div></div></td></tr></tbody></table>

4.  Select the type of project to create by selecting either the **Empty Project** or **Template or
    Example** card.

5.  Select the **Mule runtime** and **Java Version** for your app.

6.  Click **Create Project**.

## Add Components to Your Project

Add components to your integration project to build a flow:

1.  In the **Explorer** view, open the XML file for your project, such as `my-project-name.xml`.

2.  Select **Build a Flow** from the start card to create an empty flow:

    ![Anypoint Code Builder canvas with starting cards with the options to start from scratch, build a flow, or build a subflow](/mulesoft/current/en-us/connectors/_images/acb-build-flow.png)

3.  Change the name of the flow:

    1.  Click **Flow**.

    2.  Click the edit icon.

        ![Arrow pointing to the flow name edit pencil icon](/mulesoft/current/en-us/connectors/_images/acb-change-flow-name.png)

    3.  Enter the flow name, and click the checkmark.

    4.  In the canvas UI, click the
        ![Add component icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-plus.png) (**Add component**)
        icon.

    5.  In the **Add Component** panel, search for and select your component from the results. The
        following example shows the **Listener** component from the **HTTP** search results:

        ![Listener component highlighted in the Add Component section](/mulesoft/current/en-us/anypoint-code-builder/_images/main-tutorial-add-first-listener.png)

        The configuration XML file now includes the XML for the HTTP Listener into the `<flow/>`
        element, for example:

        ```XML
        <flow name="getFlights" >
          <http:listener path="" config-ref="" doc:name="Listener" doc:id="ojzfry" />
        </flow>
        ```

## Configure the Trigger For the Flow

A trigger (source) starts the flow when specified conditions are met. If a connector has its own
associated sources, you can use a connector-specific trigger to initiate the flow. Each connector
trigger has a unique configuration.

You can configure the following sources for this connector:

- **HTTP > Listener**

  Initiates a flow each time it receives a request on the configured host and port

- **Scheduler**

  Initiates a flow when a time-based condition is met

For example, to configure an HTTP Listener trigger, follow these steps:

1.  Click the **Listener** component on the canvas UI.

2.  (Optional) Click the edit icon to change the name of the **Listener** component.

3.  (Optional) If you created a configuration file for the connector, select it.

4.  In the **General** tab, in the **Path** field, enter the relative path for the path set in the
    HTTP configuration element. See
    [Configure Paths](https://docs.mulesoft.com/http-connector/latest/http-listener-ref#configure-paths).

5.  Select the **Advanced** tab to configure the following attributes:

    <table><colgroup><col> <col> <col></colgroup><thead><tr><th>Attribute</th><th>Description</th><th>Required</th></tr></thead><tbody><tr><td><p>Allowed methods</p></td><td><p>To allow all HTTP methods (default), do not define this attribute. To restrict certain methods, specify all allowed HTTP methods, for example, <code>GET, POST</code>, as a comma-separated list.</p></td><td><p>No</p></td></tr><tr><td><p>Response streaming mode</p></td><td><p>Whether to use streaming when the response is sent.</p></td><td><p>No</p></td></tr><tr><td><p>Output MIME type</p></td><td><p>MIME type of the payload that this component outputs.</p></td><td><p>No</p></td></tr><tr><td><p>Parameters</p></td><td><p>Appends a key-value pair for a reader property to the value of the Output MIME attribute. Multiple key-value pairs are allowed.</p></td><td><p>No</p></td></tr><tr><td><p>Output encoding</p></td><td><p>Encoding of the payload that this component outputs.</p></td><td><p>No</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><p>Configures the way data streams are consumed in Mule apps.</p></td><td><p>No</p></td></tr><tr><td><p>Primary node only</p></td><td><p>Select this option to execute the <strong>Listener</strong> trigger on the primary node only when running in a cluster.</p></td><td><p>No</p></td></tr><tr><td><p>Redelivery policy</p></td><td><p>Configures the redelivery policy for executing requests that generate errors. You can add a redelivery policy to any source in a flow.</p></td><td><p>No</p></td></tr><tr><td><p>Reconnection strategy</p></td><td><p>When an operation in a Mule application fails to connect to an external server, the default behavior is for the operation to fail immediately and return a connectivity error. You can modify the default behavior by configuring the reconnection strategy for the operation.</p></td><td><p>No</p></td></tr><tr><td><p>Response</p></td><td><p>See <a href="https://docs.mulesoft.com/http-connector/latest/http-documentation#HttpResponseAttributes">HTTP Response Attributes</a>.</p></td><td><p>No</p></td></tr><tr><td><p>Error Response</p></td><td><p>HTTP error response returns a status code for the error.</p></td><td><p>No</p></td></tr><tr><td><p>Body</p></td><td><p>Body of the error message.</p></td><td><p>No</p></td></tr><tr><td><p>Headers</p></td><td><p>Headers to include in the error message.</p></td><td><p>No</p></td></tr><tr><td><p>Status code</p></td><td><p>Status code of the error response, for example, <code>500</code>.</p></td><td><p>No</p></td></tr><tr><td><p>Reason phrase</p></td><td><p>Reason for the error, for example, <code>Gateway Timeout</code> or <code>Internal Server Error</code>.</p></td><td><p>No</p></td></tr></tbody></table>

## Add the Connector to Your Integration Project

1.  In the Explorer view, open the configuration XML file for your project, such as
    `my-project-name.xml`.

2.  Click the
    ![Show Mule graphical mode icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-tree-view.png) (**Show
    Mule graphical mode**) icon in the activity bar to open the canvas UI if it doesn’t open
    automatically.

3.  Add the connector the same way you added other components from the canvas UI:

    1.  In the canvas UI, click the
        ![Add component icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-plus.png) (**Add component**)
        icon.

    2.  In the **Add Component** panel, click **Connectors**.

4.  Click the
    ![Search in Exchange icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-search-exchange.png)
    (**Search in Exchange**) toggle:

    ![Search in Exchange toggle](/mulesoft/current/en-us/anypoint-code-builder/_images/int-connector-search-exchange.png)

5.  Enter `mulesoft inference` in the search box.

6.  Select the connector to add it to your project.

Adding a connector to an integration project does not make that connector available to other
projects in your Anypoint Code Builder workspace.

## Add a Global Configuration

1.  In the configuration XML, place your cursor before the `<flow/>` element and type
    `ms-inference`.

    Ensure that the cursor is not inside the `<flow/>` element.

2.  Select the snippet. Here are some examples (using OpenAI) based on the available configurations:

    ```xml
    <ms-inference:text-generation-config name="OpenAITextGen" doc:name="MuleSoft Inference Text generation config" doc:id="1ef66d01-d3ec-442a-bf61-99919d73f97e" >
    <ms-inference:openai-connection apiKey="${openai.apiKey}" openAIModelName="${openai.llmModel}" timeout="60000" />
    </ms-inference:text-generation-config>
    ```

    ```xml
    <ms-inference:moderation-config name="OpenAIModeration" doc:name="MuleSoft Inference Moderation config" doc:id="8bd9c1fe-3176-4319-9d91-090e4e6da3cc" >
        <ms-inference:openai-moderation-connection apiKey="${openai.apiKey}" openAIModelName="${openai.moderation}"/>
    </ms-inference:moderation-config>
    ```

    ```xml
    <ms-inference:image-generation-config name="OpenAIImageGen" doc:name="MuleSoft Inference Image generation config" doc:id="aba51427-1dd3-4915-a411-8fe6543ee6ac" >
        <ms-inference:openai-image-connection apiKey="${openai.apiKey}" openAIModelName="${openai.imageGenModel}" />
    </ms-inference:image-generation-config>
    ```

    ```xml
    <ms-inference:vision-config name="OpenAIVision" doc:name="MuleSoft Inference Vision config" doc:id="797661b2-a4ac-493d-8550-45d05d03da17" >
        <ms-inference:openai-vision-connection apiKey="${openai.apiKey}" openAIModelName="${openai.visionModel}" />
    </ms-inference:vision-config>
    ```

3.  Add the attributes to complete the configuration.

## Configure Provider-Specific Settings

Based on your chosen inference provider, you may need to configure additional settings. For example:

```xml
<ms-inference:text-generation-config
    provider="OPENAI"
    modelName="gpt-4o"
    temperature="0.7"
    maxTokens="1000"
    timeout="60"
    name="OpenAIConfig">
</ms-inference:text-generation-config>
```

## Configure Additional Connector Fields

After you configure a global element for the connector, configure the other required fields. The
required fields vary depending on which connector actions you use.

For more information about configuring MuleSoft Inference Connector actions, see
[MuleSoft Inference Connector - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-additional-config).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-additional-config)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=f4ac7b2e23aede93a09dfc40585e31a94a377736228af69dfc2ceb2521f26ae9 url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/mulesoft-inference-connector-config-acb.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# Using Anypoint Code Builder to Configure MuleSoft Inference Connector 1.2

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Cloud IDE (Beta) is scheduled for retirement on August 31, 2026. August 30, 2026 is the last day you
can access Cloud IDE (Beta). Download your projects to work from Desktop IDE. Learn more in
[Migrate a Project from Cloud IDE to Desktop IDE](https://docs.mulesoft.com/anypoint-code-builder/int-import-mule-project#migrate-cloud-to-desktop).

To use Anypoint Code Builder to configure a connector or module, create a basic integration project
in Anypoint Code Builder, add connectors and other components to your Mule application to process
your business logic, and configure the attributes.

## Before You Begin

Before creating an integration project, you must:

- [Set up and access the Anypoint Code Builder web or desktop IDE](https://docs.mulesoft.com/anypoint-code-builder/start-acb)

- Have credentials to access the connector’s API

- Ensure that the Visual Studio Code Extension Pack for Java is installed.

To use this connector, you must be familiar with:

- The connector’s API

- Anypoint Connectors

- Mule runtime engine (Mule)

## Create a New Integration Project

To create a new integration project:

1.  In the activity bar of the IDE, click the
    ![Anypoint Code Builder icon](/mulesoft/current/en-us/connectors/_images/acb-icon-ms-logo.png) (**Anypoint Code
    Builder**) icon.

2.  From **Quick Actions**, click **Develop an Integration**:

    ![Develop an Integration link highlighted in the Getting started section](/mulesoft/current/en-us/anypoint-code-builder/_images/develop-integration-option-mat.png)

3.  Complete the following in the **Develop an Integration** form:

    <table><colgroup><col> <col></colgroup><thead><tr><th>Attribute</th><th>Value</th></tr></thead><tbody><tr><td><div><div><p><strong>Project Name</strong></p></div></div></td><td><div><div><p>Unique name for your project.</p></div><div><p>This name is used as the title and name of the integration project file. For example, if the project name is "Salesforce Integration," the project file name is <code>salesforce-integration</code>.</p></div></div></td></tr><tr><td><div><div><p><strong>Project Location</strong></p></div></div></td><td><div><div><p>Your home directory or another directory you create. Click <strong>Browse</strong> to select a different directory for the integration project.</p></div><div><p>Don’t create the project within another project directory.</p></div><div><p>See <a href="https://docs.mulesoft.com/anypoint-code-builder/start-add-folders">Adding Folders to Your Home Directory</a>.</p></div></div></td></tr></tbody></table>

4.  Select the type of project to create by selecting either the **Empty Project** or **Template or
    Example** card.

5.  Select the **Mule runtime** and **Java Version** for your app.

6.  Click **Create Project**.

## Add Components to Your Project

Add components to your integration project to build a flow:

1.  In the **Explorer** view, open the XML file for your project, such as `my-project-name.xml`.

2.  Select **Build a Flow** from the start card to create an empty flow:

    ![Anypoint Code Builder canvas with starting cards with the options to start from scratch, build a flow, or build a subflow](/mulesoft/current/en-us/connectors/_images/acb-build-flow.png)

3.  Change the name of the flow:

    1.  Click **Flow**.

    2.  Click the edit icon.

        ![Arrow pointing to the flow name edit pencil icon](/mulesoft/current/en-us/connectors/_images/acb-change-flow-name.png)

    3.  Enter the flow name, and click the checkmark.

    4.  In the canvas UI, click the
        ![Add component icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-plus.png) (**Add component**)
        icon.

    5.  In the **Add Component** panel, search for and select your component from the results. The
        following example shows the **Listener** component from the **HTTP** search results:

        ![Listener component highlighted in the Add Component section](/mulesoft/current/en-us/anypoint-code-builder/_images/main-tutorial-add-first-listener.png)

        The configuration XML file now includes the XML for the HTTP Listener into the `<flow/>`
        element, for example:

        ```XML
        <flow name="getFlights" >
          <http:listener path="" config-ref="" doc:name="Listener" doc:id="ojzfry" />
        </flow>
        ```

## Configure the Trigger For the Flow

A trigger (source) starts the flow when specified conditions are met. If a connector has its own
associated sources, you can use a connector-specific trigger to initiate the flow. Each connector
trigger has a unique configuration.

You can configure the following sources for this connector:

- **HTTP > Listener**

  Initiates a flow each time it receives a request on the configured host and port

- **Scheduler**

  Initiates a flow when a time-based condition is met

For example, to configure an HTTP Listener trigger, follow these steps:

1.  Click the **Listener** component on the canvas UI.

2.  (Optional) Click the edit icon to change the name of the **Listener** component.

3.  (Optional) If you created a configuration file for the connector, select it.

4.  In the **General** tab, in the **Path** field, enter the relative path for the path set in the
    HTTP configuration element. See
    [Configure Paths](https://docs.mulesoft.com/http-connector/latest/http-listener-ref#configure-paths).

5.  Select the **Advanced** tab to configure the following attributes:

    <table><colgroup><col> <col> <col></colgroup><thead><tr><th>Attribute</th><th>Description</th><th>Required</th></tr></thead><tbody><tr><td><p>Allowed methods</p></td><td><p>To allow all HTTP methods (default), do not define this attribute. To restrict certain methods, specify all allowed HTTP methods, for example, <code>GET, POST</code>, as a comma-separated list.</p></td><td><p>No</p></td></tr><tr><td><p>Response streaming mode</p></td><td><p>Whether to use streaming when the response is sent.</p></td><td><p>No</p></td></tr><tr><td><p>Output MIME type</p></td><td><p>MIME type of the payload that this component outputs.</p></td><td><p>No</p></td></tr><tr><td><p>Parameters</p></td><td><p>Appends a key-value pair for a reader property to the value of the Output MIME attribute. Multiple key-value pairs are allowed.</p></td><td><p>No</p></td></tr><tr><td><p>Output encoding</p></td><td><p>Encoding of the payload that this component outputs.</p></td><td><p>No</p></td></tr><tr><td><p>Streaming Strategy</p></td><td><p>Configures the way data streams are consumed in Mule apps.</p></td><td><p>No</p></td></tr><tr><td><p>Primary node only</p></td><td><p>Select this option to execute the <strong>Listener</strong> trigger on the primary node only when running in a cluster.</p></td><td><p>No</p></td></tr><tr><td><p>Redelivery policy</p></td><td><p>Configures the redelivery policy for executing requests that generate errors. You can add a redelivery policy to any source in a flow.</p></td><td><p>No</p></td></tr><tr><td><p>Reconnection strategy</p></td><td><p>When an operation in a Mule application fails to connect to an external server, the default behavior is for the operation to fail immediately and return a connectivity error. You can modify the default behavior by configuring the reconnection strategy for the operation.</p></td><td><p>No</p></td></tr><tr><td><p>Response</p></td><td><p>See <a href="https://docs.mulesoft.com/http-connector/latest/http-documentation#HttpResponseAttributes">HTTP Response Attributes</a>.</p></td><td><p>No</p></td></tr><tr><td><p>Error Response</p></td><td><p>HTTP error response returns a status code for the error.</p></td><td><p>No</p></td></tr><tr><td><p>Body</p></td><td><p>Body of the error message.</p></td><td><p>No</p></td></tr><tr><td><p>Headers</p></td><td><p>Headers to include in the error message.</p></td><td><p>No</p></td></tr><tr><td><p>Status code</p></td><td><p>Status code of the error response, for example, <code>500</code>.</p></td><td><p>No</p></td></tr><tr><td><p>Reason phrase</p></td><td><p>Reason for the error, for example, <code>Gateway Timeout</code> or <code>Internal Server Error</code>.</p></td><td><p>No</p></td></tr></tbody></table>

## Add the Connector to Your Integration Project

1.  In the Explorer view, open the configuration XML file for your project, such as
    `my-project-name.xml`.

2.  Click the
    ![Show Mule graphical mode icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-tree-view.png) (**Show
    Mule graphical mode**) icon in the activity bar to open the canvas UI if it doesn’t open
    automatically.

3.  Add the connector the same way you added other components from the canvas UI:

    1.  In the canvas UI, click the
        ![Add component icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-plus.png) (**Add component**)
        icon.

    2.  In the **Add Component** panel, click **Connectors**.

4.  Click the
    ![Search in Exchange icon](/mulesoft/current/en-us/anypoint-code-builder/_images/icon-search-exchange.png)
    (**Search in Exchange**) toggle:

    ![Search in Exchange toggle](/mulesoft/current/en-us/anypoint-code-builder/_images/int-connector-search-exchange.png)

5.  Enter `mulesoft inference` in the search box.

6.  Select the connector to add it to your project.

Adding a connector to an integration project does not make that connector available to other
projects in your Anypoint Code Builder workspace.

## Add a Global Configuration

1.  In the configuration XML, place your cursor before the `<flow/>` element and type
    `ms-inference`.

    Ensure that the cursor is not inside the `<flow/>` element.

2.  Select the snippet. Here are some examples (using OpenAI) based on the available configurations:

    ```xml
    <ms-inference:text-generation-config name="OpenAITextGen" doc:name="MuleSoft Inference Text generation config" doc:id="1ef66d01-d3ec-442a-bf61-99919d73f97e" >
    <ms-inference:openai-connection apiKey="${openai.apiKey}" openAIModelName="${openai.llmModel}" timeout="60000" />
    </ms-inference:text-generation-config>
    ```

    ```xml
    <ms-inference:moderation-config name="OpenAIModeration" doc:name="MuleSoft Inference Moderation config" doc:id="8bd9c1fe-3176-4319-9d91-090e4e6da3cc" >
        <ms-inference:openai-moderation-connection apiKey="${openai.apiKey}" openAIModelName="${openai.moderation}"/>
    </ms-inference:moderation-config>
    ```

    ```xml
    <ms-inference:image-generation-config name="OpenAIImageGen" doc:name="MuleSoft Inference Image generation config" doc:id="aba51427-1dd3-4915-a411-8fe6543ee6ac" >
        <ms-inference:openai-image-connection apiKey="${openai.apiKey}" openAIModelName="${openai.imageGenModel}" />
    </ms-inference:image-generation-config>
    ```

    ```xml
    <ms-inference:vision-config name="OpenAIVision" doc:name="MuleSoft Inference Vision config" doc:id="797661b2-a4ac-493d-8550-45d05d03da17" >
        <ms-inference:openai-vision-connection apiKey="${openai.apiKey}" openAIModelName="${openai.visionModel}" />
    </ms-inference:vision-config>
    ```

3.  Add the attributes to complete the configuration.

## Configure Provider-Specific Settings

Based on your chosen inference provider, you may need to configure additional settings. For example:

```xml
<ms-inference:text-generation-config
    provider="OPENAI"
    modelName="gpt-4o"
    temperature="0.7"
    maxTokens="1000"
    timeout="60"
    name="OpenAIConfig">
</ms-inference:text-generation-config>
```

## Configure Additional Connector Fields

After you configure a global element for the connector, configure the other required fields. The
required fields vary depending on which connector actions you use.

For more information about configuring MuleSoft Inference Connector actions, see
[MuleSoft Inference Connector - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-additional-config).

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

- [MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-additional-config)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=2de5079a96c59932824b56680776927b99d48630f299a99126458d3a563e2715 url=/mulesoft/current/en-us/mulesoft-inference-connector/1.0/mulesoft-inference-connector-studio.md -->

---
product: MuleSoft Inference Connector
version: 1.0
---

# Using Anypoint Studio to Configure MuleSoft Inference Connector 1.0

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Anypoint Studio (Studio) editors help you design and update your Mule applications, properties, and
configuration files.

To add and configure a connector in Studio:

1.  [Create a Mule project](#create-mule-project).

2.  [Add the connector to your Mule project](#add-connector-to-project).

3.  [Configure a source for the connector’s flow](#configure-source).

4.  [Add a connector operation to the flow](#add-connector-operation).

5.  [Configure a global element for the connector](#configure-global-element).

6.  [Configure the other connector fields](#configure-other-fields).

When you run the connector, you can view the app log to check for problems in real time, as
described in [View the App Log](#view-app-log).

If you are new to configuring connectors in Studio, see
[Using Anypoint Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio).
If, after reading this topic, you need additional information about the connector fields, see the
[MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference).

## Create a Mule Project

In Studio, create a new Mule project in which to add and configure Anypoint Connector for MuleSoft
Inference (MuleSoft Inference Connector):

1.  In Studio, select **File > New > Mule Project**.

2.  Enter a name for your Mule project and click **Finish**.

## Add the Connector to Your Mule Project

Add MuleSoft Inference Connector to your Mule project to automatically populate the XML code with
the connector’s namespace and schema location and add the required dependencies to the project’s
`pom.xml` file:

1.  In **Mule Palette**, click **(X) Search in Exchange**.

2.  In **Add Dependencies to Project**, type `mulesoft inference` in the search field.

3.  Click **MuleSoft Inference Connector** in **Available modules**.

4.  Click **Add**.

5.  Click **Finish**.

Adding a connector to a Mule project in Studio does not make that connector available to other
projects in your Studio workspace.

## Configure a Source

A source initiates a flow when a specified condition is met. You can configure one of these sources
to use with MuleSoft Inference Connector:

- **HTTP > Listener**

  Initiates a flow each time it receives a request on the configured host and port

- **Scheduler**

  Initiates a flow when a time-based condition is met

For example, to configure an **HTTP > Listener** source, follow these steps:

1.  In **Mule Palette**, select **HTTP > Listener**.

2.  Drag **Listener** to the Studio canvas.

3.  On the **Listener** configuration screen, optionally change the value of the **Display Name**
    field.

4.  Specify a value for the **Path** field.

5.  Click the plus sign (**+**) next to the **Connector configuration** field to configure a global
    element that can be used by all instances of the **HTTP > Listener** source in the app.

6.  On the **General** tab, specify the connection information for the connector.

7.  On the **TLS** tab, optionally specify the TLS information for the connector.

8.  On the **Advanced** tab, optionally specify reconnection information, including a reconnection
    strategy.

9.  Click **Test Connection** to confirm that Mule can connect with the specified server.

10. Click **OK**.

## Add a Connector Operation to the Flow

When you add a connector operation to your flow, you are specifying an action for that connector to
perform.

To add an operation for MuleSoft Inference Connector, follow these steps:

1.  In **Mule Palette**, select **MuleSoft Inference Connector** and then select the operation to
    add.

2.  Drag the operation onto the Studio canvas, next to the source.

## Configure a Global Element for the Connector

When you configure a connector, configure a global element that all instances of that connector in
the app can use. Configuring a global element requires you to provide the authentication credentials
that the connector requires to access the target inference provider.

You can reference a configuration file that contains ANT-style property placeholders (recommended),
or you can enter your authorization credentials in the global configuration properties. For
information about the benefits of using property placeholders and how to configure them, see
[Anypoint Connector Configuration](https://docs.mulesoft.com/connectors/introduction/intro-connector-configuration-overview).

To configure the global element for MuleSoft Inference Connector, follow these steps.

1.  Select the operation in the Studio canvas.

2.  On the properties screen for the operation, click the Add (**+**) icon to access the global
    element configuration fields.

3.  Choose from these configs.

    - Image Generation Config

    - Moderation Config

    - Text Generation Config

    - Vision Config

4.  Fill out the fields depending on the inference type.

5.  Click **OK**.

## Configure Additional Connector Fields

After you configure a global element for the connector, configure the other required fields. The
required fields vary depending on which connector operation you use.

For detailed configuration information for each operation type, see
[MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-additional-config).

## View the App Log

To check for problems, you can view the app log as follows:

- If you’re running the app from Anypoint Platform, the app log output is visible on the Anypoint
  Studio console window.

- If you’re running the app using Mule from the command line, the app log output is visible in your
  operating system console.

Unless the log file path is customized in the app’s log file (`log4j2.xml`), you can also view the
app log in the default location `MULE_HOME/logs/<app-name>.log`. You can configure the location of
the log path in the app log file `log4j2.xml`.

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Using Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.0/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=ec0c9d2d46698bbf8d6ec4465637db4521ed84239bbeef59eeaf8bf215bdb55c url=/mulesoft/current/en-us/mulesoft-inference-connector/1.1/mulesoft-inference-connector-studio.md -->

---
product: MuleSoft Inference Connector
version: 1.1
---

# Using Anypoint Studio to Configure MuleSoft Inference Connector 1.1

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Anypoint Studio (Studio) editors help you design and update your Mule applications, properties, and
configuration files.

To add and configure a connector in Studio:

1.  [Create a Mule project](#create-mule-project).

2.  [Add the connector to your Mule project](#add-connector-to-project).

3.  [Configure a source for the connector’s flow](#configure-source).

4.  [Add a connector operation to the flow](#add-connector-operation).

5.  [Configure a global element for the connector](#configure-global-element).

6.  [Configure the other connector fields](#configure-other-fields).

When you run the connector, you can view the app log to check for problems in real time, as
described in [View the App Log](#view-app-log).

If you are new to configuring connectors in Studio, see
[Using Anypoint Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio).
If, after reading this topic, you need additional information about the connector fields, see the
[MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference).

## Create a Mule Project

In Studio, create a new Mule project in which to add and configure Anypoint Connector for MuleSoft
Inference (MuleSoft Inference Connector):

1.  In Studio, select **File > New > Mule Project**.

2.  Enter a name for your Mule project and click **Finish**.

## Add the Connector to Your Mule Project

Add MuleSoft Inference Connector to your Mule project to automatically populate the XML code with
the connector’s namespace and schema location and add the required dependencies to the project’s
`pom.xml` file:

1.  In **Mule Palette**, click **(X) Search in Exchange**.

2.  In **Add Dependencies to Project**, type `mulesoft inference` in the search field.

3.  Click **MuleSoft Inference Connector** in **Available modules**.

4.  Click **Add**.

5.  Click **Finish**.

Adding a connector to a Mule project in Studio does not make that connector available to other
projects in your Studio workspace.

## Configure a Source

A source initiates a flow when a specified condition is met. You can configure one of these sources
to use with MuleSoft Inference Connector:

- **HTTP > Listener**

  Initiates a flow each time it receives a request on the configured host and port

- **Scheduler**

  Initiates a flow when a time-based condition is met

For example, to configure an **HTTP > Listener** source, follow these steps:

1.  In **Mule Palette**, select **HTTP > Listener**.

2.  Drag **Listener** to the Studio canvas.

3.  On the **Listener** configuration screen, optionally change the value of the **Display Name**
    field.

4.  Specify a value for the **Path** field.

5.  Click the plus sign (**+**) next to the **Connector configuration** field to configure a global
    element that can be used by all instances of the **HTTP > Listener** source in the app.

6.  On the **General** tab, specify the connection information for the connector.

7.  On the **TLS** tab, optionally specify the TLS information for the connector.

8.  On the **Advanced** tab, optionally specify reconnection information, including a reconnection
    strategy.

9.  Click **Test Connection** to confirm that Mule can connect with the specified server.

10. Click **OK**.

## Add a Connector Operation to the Flow

When you add a connector operation to your flow, you are specifying an action for that connector to
perform.

To add an operation for MuleSoft Inference Connector, follow these steps:

1.  In **Mule Palette**, select **MuleSoft Inference Connector** and then select the operation to
    add.

2.  Drag the operation onto the Studio canvas, next to the source.

## Configure a Global Element for the Connector

When you configure a connector, configure a global element that all instances of that connector in
the app can use. Configuring a global element requires you to provide the authentication credentials
that the connector requires to access the target inference provider.

You can reference a configuration file that contains ANT-style property placeholders (recommended),
or you can enter your authorization credentials in the global configuration properties. For
information about the benefits of using property placeholders and how to configure them, see
[Anypoint Connector Configuration](https://docs.mulesoft.com/connectors/introduction/intro-connector-configuration-overview).

To configure the global element for MuleSoft Inference Connector, follow these steps.

1.  Select the operation in the Studio canvas.

2.  On the properties screen for the operation, click the Add (**+**) icon to access the global
    element configuration fields.

3.  Choose from these configs.

    - Image Generation Config

    - Moderation Config

    - Text Generation Config

    - Vision Config

4.  Fill out the fields depending on the inference type.

5.  Click **OK**.

## Configure Additional Connector Fields

After you configure a global element for the connector, configure the other required fields. The
required fields vary depending on which connector operation you use.

For detailed configuration information for each operation type, see
[MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-additional-config).

## View the App Log

To check for problems, you can view the app log as follows:

- If you’re running the app from Anypoint Platform, the app log output is visible on the Anypoint
  Studio console window.

- If you’re running the app using Mule from the command line, the app log output is visible in your
  operating system console.

Unless the log file path is customized in the app’s log file (`log4j2.xml`), you can also view the
app log in the default location `MULE_HOME/logs/<app-name>.log`. You can configure the location of
the log path in the app log file `log4j2.xml`.

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Using Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/1.1/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)

<!-- document id=d0b368bd021d7c71c656b20755df19a6fa736e453d3df8ab8e79038e1be67a7f url=/mulesoft/current/en-us/mulesoft-inference-connector/latest/mulesoft-inference-connector-studio.md -->

---
product: MuleSoft Inference Connector
version: 1.2
is-latest-version: true
---

# Using Anypoint Studio to Configure MuleSoft Inference Connector 1.2

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Anypoint Studio (Studio) editors help you design and update your Mule applications, properties, and
configuration files.

To add and configure a connector in Studio:

1.  [Create a Mule project](#create-mule-project).

2.  [Add the connector to your Mule project](#add-connector-to-project).

3.  [Configure a source for the connector’s flow](#configure-source).

4.  [Add a connector operation to the flow](#add-connector-operation).

5.  [Configure a global element for the connector](#configure-global-element).

6.  [Configure the other connector fields](#configure-other-fields).

When you run the connector, you can view the app log to check for problems in real time, as
described in [View the App Log](#view-app-log).

If you are new to configuring connectors in Studio, see
[Using Anypoint Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio).
If, after reading this topic, you need additional information about the connector fields, see the
[MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference).

## Create a Mule Project

In Studio, create a new Mule project in which to add and configure Anypoint Connector for MuleSoft
Inference (MuleSoft Inference Connector):

1.  In Studio, select **File > New > Mule Project**.

2.  Enter a name for your Mule project and click **Finish**.

## Add the Connector to Your Mule Project

Add MuleSoft Inference Connector to your Mule project to automatically populate the XML code with
the connector’s namespace and schema location and add the required dependencies to the project’s
`pom.xml` file:

1.  In **Mule Palette**, click **(X) Search in Exchange**.

2.  In **Add Dependencies to Project**, type `mulesoft inference` in the search field.

3.  Click **MuleSoft Inference Connector** in **Available modules**.

4.  Click **Add**.

5.  Click **Finish**.

Adding a connector to a Mule project in Studio does not make that connector available to other
projects in your Studio workspace.

## Configure a Source

A source initiates a flow when a specified condition is met. You can configure one of these sources
to use with MuleSoft Inference Connector:

- **HTTP > Listener**

  Initiates a flow each time it receives a request on the configured host and port

- **Scheduler**

  Initiates a flow when a time-based condition is met

For example, to configure an **HTTP > Listener** source, follow these steps:

1.  In **Mule Palette**, select **HTTP > Listener**.

2.  Drag **Listener** to the Studio canvas.

3.  On the **Listener** configuration screen, optionally change the value of the **Display Name**
    field.

4.  Specify a value for the **Path** field.

5.  Click the plus sign (**+**) next to the **Connector configuration** field to configure a global
    element that can be used by all instances of the **HTTP > Listener** source in the app.

6.  On the **General** tab, specify the connection information for the connector.

7.  On the **TLS** tab, optionally specify the TLS information for the connector.

8.  On the **Advanced** tab, optionally specify reconnection information, including a reconnection
    strategy.

9.  Click **Test Connection** to confirm that Mule can connect with the specified server.

10. Click **OK**.

## Add a Connector Operation to the Flow

When you add a connector operation to your flow, you are specifying an action for that connector to
perform.

To add an operation for MuleSoft Inference Connector, follow these steps:

1.  In **Mule Palette**, select **MuleSoft Inference Connector** and then select the operation to
    add.

2.  Drag the operation onto the Studio canvas, next to the source.

## Configure a Global Element for the Connector

When you configure a connector, configure a global element that all instances of that connector in
the app can use. Configuring a global element requires you to provide the authentication credentials
that the connector requires to access the target inference provider.

You can reference a configuration file that contains ANT-style property placeholders (recommended),
or you can enter your authorization credentials in the global configuration properties. For
information about the benefits of using property placeholders and how to configure them, see
[Anypoint Connector Configuration](https://docs.mulesoft.com/connectors/introduction/intro-connector-configuration-overview).

To configure the global element for MuleSoft Inference Connector, follow these steps.

1.  Select the operation in the Studio canvas.

2.  On the properties screen for the operation, click the Add (**+**) icon to access the global
    element configuration fields.

3.  Choose from these configs.

    - Image Generation Config

    - Moderation Config

    - Text Generation Config

    - Vision Config

4.  Fill out the fields depending on the inference type.

5.  Click **OK**.

## Configure Additional Connector Fields

After you configure a global element for the connector, configure the other required fields. The
required fields vary depending on which connector operation you use.

For detailed configuration information for each operation type, see
[MuleSoft Inference - Additional Configuration Information](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-additional-config).

## View the App Log

To check for problems, you can view the app log as follows:

- If you’re running the app from Anypoint Platform, the app log output is visible on the Anypoint
  Studio console window.

- If you’re running the app using Mule from the command line, the app log output is visible in your
  operating system console.

Unless the log file path is customized in the app’s log file (`log4j2.xml`), you can also view the
app log in the default location `MULE_HOME/logs/<app-name>.log`. You can configure the location of
the log path in the app log file `log4j2.xml`.

## See Also

- [Introduction to Anypoint Connectors](https://docs.mulesoft.com/connectors/introduction/introduction-to-anypoint-connectors)

- [Using Studio to Configure a Connector](https://docs.mulesoft.com/connectors/introduction/intro-config-use-studio)

- [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference)

- [Salesforce Help](https://help.salesforce.com)