--- schema_version: "1" id: f4913beaa2d18a9a6f12871f79a13c6b46408ff76b452acdefab0ed034a974a1 title: Configuring Text Generation Operations collection: mulesoft version: current locale: en-us portal_url: https://docs.salesforce.com/mulesoft/current/en-us/mulesoft-inference-connector/latest/configuring-text-generation-operations.md canonical_url: https://docs.mulesoft.com/mulesoft-inference-connector/latest/configuring-text-generation-operations source_site: docs.mulesoft.com content_type: documentation content_hash: 1b0178a93735e8ab32b7cd4fbc58a679a7dca4dc359edacbd9f44fbdbae7b7fe description: MuleSoft Documentation Site release: "1.2" latest: true --- --- product: MuleSoft Inference Connector version: 1.2 is-latest-version: true --- # Configuring Text Generation Operations > For the full documentation index, see: https://docs.mulesoft.com/llms.txt Configure the **\[Agent\] Define Prompt Template**, **\[Chat\] Answer Prompt**, **\[Chat\] Completions**, **\[Tools\] Native Template (Reasoning only)**, and **\[MCP\] Tooling** operations. ## Configure the Agent Define Prompt Template Operation The **\[Agent\] Define Prompt Template** operation is essential for configuring specific prompt templates with the LLM of your choice. This operation allows you to define and compose AI functions using plain text, enabling the creation of natural language prompts, generating responses, extracting information, invoking other prompts, or performing any text-based task. Apply the **\[Agent\] Define Prompt Template** operation in various scenarios, such as for: - **Customer Service Agents** Enhance customer service by providing case summaries, case classifications, summarizing large datasets, and more. - **Sales Operation Agents** Assist sales teams in writing sales emails, summarizing cases for specific accounts, assessing the probability of closing deals, and more. - **Marketing Agents** Support marketing teams in generating product descriptions, creating newsletters, planning social media campaigns, and more. To configure the **\[Agent\] Define Prompt Template** operation: 1. Select the operation on the Anypoint Code Builder or Studio canvas. 2. In the **General** properties tab for the operation, enter these values: - **Template** Enter the prompt template for the operation. - **Instructions** Provide the instructions for the inference provider, outlining the goals of the task. - **Data** Specify the data for the inference provider to evaluate using the provided template and instructions. 3. In the **Additional Request Attributes** (optional) field, you can pass any additional request attributes in the request payload to generate more relevant and precise LLM outputs. The corresponding XML DSL tag for this operation is ``. This is the XML for this operation: ```xml ``` ### Output Configuration This operation responds with a JSON payload containing the main LLM response. This is an example response: ```json { "response": "{\n \"type\": \"positive\",\n \"response\": \"Thank you for your positive feedback on the training last week. We are glad to hear that you had a great experience. Have a nice day!\"\n}" } ``` The operation also returns attributes that aren’t within the main JSON payload, that include information about token usage, for example: ```json { "attributes": { "tokenUsage": { "inputCount": 12, "outputCount": 350, "totalCount": 362 }, "additionalAttributes": { "finishReason": "stop", "model": "openai/gpt-4o-mini", "id": "agentcmpl-fc4137f6-0b40-4018-936f-f0df6c0b5da1", "promptFilterResults": [ { "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "jailbreak": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } }, "promptIndex": 0 } ], "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "protected_material_code": { "severity": null, "filtered": false, "detected": false }, "protected_material_text": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } } } } } ``` - `tokenUsage`: Token usage metadata returned as attributes - `inputCount`: Number of tokens used to process the input - `outputCount`: Number of tokens used to generate the output - `totalCount`: Total number of tokens used for input and output - `additionalAttributes`: Additional metadata from the LLM provider - `finish_reason`: The finish reason for the LLM response - `model`: The ID of the model used - `id`: The ID of the request - `promptFilterResults`: Flags if the user’s input contains potentially harmful content - `contentFilterResults`: Flags if the generated response contains potentially harmful content ## Configure the Chat Answer Prompt Operation The **\[Chat\] Answer Prompt** operation is a simple prompt request operation to the configured LLM. It uses a plain text prompt as input and responds with a plain text answer. Apply the **\[Chat\] Answer Prompt** operation in various scenarios, such as for: - **Basic Chatbots** Answer simple user prompts. - **Customer Service Queries** Provide direct answers to frequently asked questions. To configure the **\[Chat\] Answer Prompt** operation: 1. Select the operation on the Anypoint Code Builder or Studio canvas. 2. In the **General** properties tab for the operation, enter these values: - **Prompt** Enter the prompt as plain text for the operation. 3. In the **Additional Request Attributes** (optional) field, you can pass any additional request attributes in the request payload to generate more relevant and precise LLM outputs. The corresponding XML DSL tag for this operation is ``. This is the XML for this operation: ```xml ``` ### Output Configuration This operation responds with a JSON payload containing the main LLM response. This is an example response: ```json { "response": "The capital of Switzerland is Bern. It's known for its well-preserved medieval old town, which is a UNESCO World Heritage site. Bern became the capital of Switzerland in 1848. The Swiss parliament, the Federal Assembly, is located in Bern." } ``` The operation also returns attributes that aren’t within the main JSON payload, that include information about token usage, for example: ```json { "attributes": { "tokenUsage": { "inputCount": 12, "outputCount": 350, "totalCount": 362 }, "additionalAttributes": { "finishReason": "stop", "model": "gpt-4o-mini", "id": "chatanswer-gc2425f6-0b70-4018-936f-f0df6c0b5da1", "promptFilterResults": [ { "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "jailbreak": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } }, "promptIndex": 0 } ], "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "protected_material_code": { "severity": null, "filtered": false, "detected": false }, "protected_material_text": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } } } } } ``` - `tokenUsage`: Token usage metadata returned as attributes - `inputCount`: Number of tokens used to process the input - `outputCount`: Number of tokens used to generate the output - `totalCount`: Total number of tokens used for input and output - `additionalAttributes`: Additional metadata from the LLM provider - `finish_reason`: The finish reason for the LLM response - `model`: The ID of the model used - `id`: The ID of the request - `promptFilterResults`: Flags if the user’s input contains potentially harmful content - `contentFilterResults`: Flags if the generated response contains potentially harmful content ## Configure the Chat Completions Operation The **\[Chat\] Completions** operation is useful when you want to retain conversation history for a multi-user chat operation. MuleSoft Object Store or any database can hold previous user conversation, which can be provided as messages into the **\[Chat\] Completions** operation. Apply the **\[Chat\] Completions** operation in various scenarios, such as for: - **Customer Support Chats** Retaining the context of ongoing support conversations. - **Multi-user Chat Applications** Maintaining conversation history for different users. - **Personal Assistants** Keeping track of user interactions to provide more relevant responses. To configure the **\[Chat\] Completions** operation: 1. Select the operation on the Anypoint Code Builder or Studio canvas. 2. In the **General** properties tab for the operation, enter these values: - **Messages** Specify the conversation history as a JSON array. This field supports DataSense, which provides schema support. 3. In the **Additional Request Attributes** (optional) field, you can pass any additional request attributes in the request payload to generate more relevant and precise LLM outputs. The corresponding XML DSL tag for this operation is ``. This is the XML for this operation: ```xml ``` > [!NOTE] You can declare the `payload.data` variable separately or you can inline it with the > messages tag. ### Output Configuration This operation responds with a JSON payload containing the main LLM response. This is an example response: ```json { "response": "The capital of Switzerland is **Bern**. 🇨🇭 \n" } ``` The operation also returns attributes that aren’t within the main JSON payload, that include information about token usage and additional metadata, for example: ```json { "attributes": { "tokenUsage": { "inputCount": 12, "outputCount": 350, "totalCount": 362 }, "additionalAttributes": { "finishReason": "stop", "model": "gpt-4o-mini", "id": "chatcmpl-fc2425f6-0b70-4018-936f-f0df6c0b5da1" "promptFilterResults": [ { "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "jailbreak": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } }, "promptIndex": 0 } ], "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "protected_material_code": { "severity": null, "filtered": false, "detected": false }, "protected_material_text": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } } } } } ``` - `tokenUsage`: Token usage metadata returned as attributes - `inputCount`: Number of tokens used to process the input - `outputCount`: Number of tokens used to generate the output - `totalCount`: Total number of tokens used for input and output - `additionalAttributes`: Additional metadata from the LLM provider - `finish_reason`: The finish reason for the LLM response - `model`: The ID of the model used - `id`: The ID of the request - `promptFilterResults`: Flags if the user’s input contains potentially harmful content - `contentFilterResults`: Flags if the generated response contains potentially harmful content ## Configure the Tools Native Template (Reasoning only) Operation The **\[Tools\] Native Template (Reasoning only)** operation is useful if you want to create autonomous agents that can use external tools whenever a prompt can’t be answered directly by the AI model. This operation selects which tools should be used for execution. Apply the **\[Tools\] Native Template (Reasoning only)** operation in various scenarios, such as for: - **Automating Routine Tasks** Create autonomous agents that handle routine tasks by calling appropriate APIs. - **Customer Support** Automate responses to common queries by integrating tools that provide necessary information. - **Inventory Management** Use tools to check inventory levels or order status based on user prompts. - **Employee Management** Retrieve employee information or manage employee-related tasks through API calls. - **Sales and Marketing** Access CRM data or manage leads and accounts efficiently using predefined tools. To configure the **\[Tools\] Native Template (Reasoning only)** operation: 1. Select the operation on the Anypoint Code Builder or Studio canvas. 2. In the **General** properties tab for the operation, enter these values: - **Template** Enter the template for the operation. - **Instructions** Enter the instructions for the operation. - **Data** Enter the prompt to send to the LLM, along the tools array. The result suggests a chaining of the tool as a request to fulfill the users request. - **Tools** Enter the tools for the operation. This field supports DataSense, which provides schema support. 3. In the **Additional Request Attributes** (optional) field, you can pass any additional request attributes in the request payload to generate more relevant and precise LLM outputs. The corresponding XML DSL tag for this operation is ``. This is an example of the tools array in the payload: ```json { "tools": [ { "type": "function", "function": { "name": "get_current_temperature", "description": "Get the current temperature for a specific location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g., San Francisco, CA" }, "unit": { "type": "string", "enum": ["Celsius", "Fahrenheit"], "description": "The temperature unit to use. Infer this from the user's location." } }, "required": ["location", "unit"] } } }, { "type": "function", "function": { "name": "get_rain_probability", "description": "Get the probability of rain for a specific location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g., San Francisco, CA" } }, "required": ["location"] } } }, { "type": "function", "function": { "name": "get_delivery_date", "description": "Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'", "parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The customer's order ID." } }, "required": ["order_id"] } } } ] } ``` This is the XML for this operation: ```xml ``` ### Output Configuration This operation responds with a JSON payload containing the main LLM response. #### Example Response Payload (Tools not used) If the prompt is a general question that can be answered with public knowledge, such as `What is the capital of Switzerland?`, the response is: ```json { "response": "The capital of Switzerland is Bern." } ``` #### Example Response Payload (Tools used) If the prompt requires accessing external data through a tool, such as `When will my order (ID = 220) be delivered?`, the AI model uses the configured tool to fetch the information, and the response is: ```json { "response": { "tools": [ { "function": { "name": "get_delivery_date", "arguments": "{\"order_id\":\"220\"}" }, "id": "call_ylw95cO0kxCt7NELy91cF7bX", "type": "function" } ] } } ``` The operation also returns attributes that aren’t within the main JSON payload, that include information about token usage, for example: ```json { "attributes": { "tokenUsage": { "inputCount": 204, "outputCount": 16, "totalCount": 220 }, "additionalAttributes": { "finishReason": "tool_calls", "model": "gpt-4o-mini", "id": "chatcmpl-AQwXGmrzcO7XG5Ic1Hp0PKIrRoL4O" "promptFilterResults": [ { "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "jailbreak": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } }, "promptIndex": 0 } ], "contentFilterResults": { "hate": { "severity": "safe", "filtered": false, "detected": null }, "protected_material_code": { "severity": null, "filtered": false, "detected": false }, "protected_material_text": { "severity": null, "filtered": false, "detected": false }, "self_harm": { "severity": "safe", "filtered": false, "detected": null }, "sexual": { "severity": "safe", "filtered": false, "detected": null }, "violence": { "severity": "safe", "filtered": false, "detected": null } } } } } ``` - `tokenUsage`: Token usage metadata returned as attributes - `inputCount`: Number of tokens used to process the input - `outputCount`: Number of tokens used to generate the output - `totalCount`: Total number of tokens used for input and output - `additionalAttributes`: Additional metadata from the LLM provider - `finish_reason`: The finish reason for the LLM response - `model`: The ID of the model used - `id`: The ID of the request - `promptFilterResults`: Flags if the user’s input contains potentially harmful content - `contentFilterResults`: Flags if the generated response contains potentially harmful content ## Configure the MCP Tooling Operation The **\[MCP\] Tooling** operation allows you to execute tools hosted on an MCP server directly through user-defined instructions. Instead of manually managing multiple connectors or workflows, you can simply state what you want to achieve, and the LLM intelligently selects and runs the most relevant tools. This selective execution model improves accuracy, reduces operational overhead, and ensures users or agents interact only with the capabilities that matter — resulting in faster execution, streamlined operations, and more predictable outcomes. Apply the **\[MCP\] Tooling** operation in various scenarios, such as for: - **IT Automation Agents** Automate routine IT tasks by executing MCP tools to trigger system diagnostics, restart services, fetch configuration data, and more. - **Data Analysis Agents** Leverage MCP tools to extract metrics from databases, run statistical models, generate visual summaries, and more. - **DevOps Agents** Streamline deployment workflows by invoking MCP tools to monitor CI/CD pipelines, retrieve logs, manage environment variables, and more. - **Procurement Agents** Use MCP-based actions to check vendor pricing, generate purchase requests, validate contract statuses, and more. - **SHR & Recruitment Agents** Automate candidate screening with MCP-enabled resume parsing, skill extraction, interview scheduling, and more. As a prerequisite, ensure you download MCP Connector from Anypoint Exchange. Note that MuleSoft Inference Connector now supports MCP Connector 1.2. To configure the **\[MCP\] Tooling** operation: 1. Select the operation on the Anypoint Code Builder or Studio canvas. 2. In the **General** properties tab for the operation, enter these values: - **Mcp config references** Enter the list of MCP client configuration references. - **Mcp Client Config Ref** Enter the reference to the MCP client configuration defined as part of configuration from the MCP connector. - **Template** Enter the prompt template for the operation. - **Instructions** Provide the instructions for the inference provider, outlining the goals of the task. - **Data** Enter the prompt to send to the LLM, along the tools array. The result suggests a chaining of the tool as a request to fulfill the users request. 3. In the **Additional Request Attributes** (optional) field, you can pass any additional request attributes in the request payload to generate more relevant and precise LLM outputs. The corresponding XML DSL tag for this operation is ``. For more information about MCP Connector, see [MCP Connector Reference](https://docs.mulesoft.com/mcp-connector/latest/mcp-connector-reference). This is the XML for this operation: ```xml ``` ### Output Configuration This operation responds with a JSON payload containing the main LLM response. This is an example response: ```json { "payload": { "response": "The item MULETEST0 has 50 units in stock in the primary warehouse. Is there anything else I can help you with?", "tool_used": "CheckInventory" }, ``` The operation also returns attributes that aren’t within the main JSON payload, that include information about token usage, for example: ```json "attributes": { "tokenUsage": { "outputCount": 27, "totalCount": 1973, "inputCount": 1879 }, "additionalAttributes": { "promptFilterResults": null, "contentFilterResults": null, "finishReason": "STOP", "model": "gemini-2.5-flash", "id": "Em3aaPrJOceBmtkPvp3DuQo" } } ``` - `tokenUsage`: Token usage metadata returned as attributes - `inputCount`: Number of tokens used to process the input - `outputCount`: Number of tokens used to generate the output - `totalCount`: Total number of tokens used for input and output - `additionalAttributes`: Additional metadata from the LLM provider - `finish_reason`: The finish reason for the LLM response - `model`: The ID of the model used - `id`: The ID of the request > [!NOTE] For the Gemini connection type, additional parameters must be included within the > `GenerationConfig` property of the request payload. ## See Also - [MuleSoft Inference Connector Overview](https://docs.mulesoft.com/mulesoft-inference-connector/latest/) - [MuleSoft Inference Connector Reference](https://docs.mulesoft.com/mulesoft-inference-connector/latest/mulesoft-inference-connector-reference) - [Salesforce Help](https://help.salesforce.com)