--- schema_version: "1" id: e8967d1c20eadbba170720376c49acf03951b8db0c112ccbbfd55064a924990d title: Custom Scorer Prompt Examples collection: admin version: current locale: en-us portal_url: https://docs.salesforce.com/admin/current/en-us/ai/264-0-0/generative_ai_optimize_scorer_examples.md canonical_url: https://help.salesforce.com/s/articleView?id=ai.generative_ai_optimize_scorer_examples.htm&release=264.0.0&type=5 source_site: help.salesforce.com content_type: documentation content_hash: a17a4e2a894c5b9852fe2a26b67ec96d138a21e69ec494446995536f1966ed4a description: Use basic and advanced prompt examples to create custom scorers for common evaluation patterns. products: - Agentforce - Cross Cloud release: "264" --- # Custom Scorer Prompt Examples Use basic and advanced prompt examples to create custom scorers for common evaluation patterns. ### Required Editions | Required Editions | | --- | | Scorers are available in Salesforce Enterprise, Performance, and Unlimited Editions with Salesforce Foundations or Agentforce 1 Edition. | ## Basic Examples Use these concise examples to understand the basic structure of a scorer prompt. For production use cases that require thresholds, fallback labels, mixed outputs, or multiple labels, see the advanced examples. **Sentiment Classification (predefined)** — Classifies overall customer sentiment by using one of three predefined labels. ```text ### Role You are a sentiment evaluation model for support sessions. ### Instructions 1. Read the full session transcript. 2. Classify overall customer sentiment using exactly one label: - Positive - Neutral - Negative 3. Do not output explanations. ### Input Data Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ### Output Format Return only one label: Positive | Neutral | Negative ``` **Containment Classification (predefined)** — Determines whether the AI agent resolved the customer's issue without escalating to a human agent. ```text ### Role You are a containment evaluator for support sessions. ### Instructions 1. Determine whether the issue was resolved by the AI agent without handoff. 2. Use exactly one label: - Contained - Not Contained 3. Return Not Contained if escalation or explicit handoff occurred. ### Input Data Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ### Output Format Return only one label: Contained | Not Contained ``` **Case ID Extraction (open-ended)** — Extracts case identifiers for tagging, tracking, or linking sessions to support cases. ```text ### Role You are a case identifier extraction model. ### Instructions 1. Extract all case IDs explicitly mentioned in the transcript. 2. Preserve each identifier exactly as written. 3. If none are present, return "none". ### Input Data Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ### Output Format Return a comma-separated list of case IDs, or "none". ``` **Customer Frustration Summary (open-ended)** — Summarizes the customer's emotional state and main pain points. ```text ### Role You are a customer frustration summarization model. ### Instructions 1. Summarize the customer's emotional state and main pain points. 2. Use 2-4 sentences in neutral, factual language. 3. Do not invent details that aren't present in the transcript. ### Input Data Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ### Output Format Return only the 2-4 sentence summary. ``` ## Advanced Examples Use these production-style examples to structure scorer output as JSON and support more complex evaluation requirements. Open-ended custom scorers return JSON that conforms to this schema: ```json { "outputs": [ { "label": "optional string", "value": "optional string", "isPassed": true } ], "explanation": "optional string" } ``` - Use `label` only for a selection from a provided list of allowed labels. - Use `value` for an output that isn't selected from an allowed-label list. - Use the optional Boolean `isPassed` when the prompt defines pass criteria. - Use `explanation` to provide supporting evidence when the prompt requests it. Adapt the criteria, labels, thresholds, and definitions in these examples to your business requirements. **Customer Sentiment: Numeric Value with a Pass Threshold** — Returns a numeric value from 1 through 5 and uses `isPassed` to indicate whether the score meets a threshold of 3. ```text You are evaluating the customer’s sentiment in a conversation between a customer and an AI agent. Read the full conversation transcript and assign one numerical sentiment rating that best represents the customer’s overall emotional attitude toward the interaction, product, service, or issue being discussed. Scorer name: Customer Sentiment Scorer description: Measures whether the customer expresses negative, neutral, or positive sentiment during the conversation. Rating Instructions: Assign exactly one integer from 1 to 5 using the following criteria: 1 - Strongly negative The customer expresses intense dissatisfaction, anger, frustration, hostility, or loss of trust. Examples include threats to cancel, repeated complaints, demands for escalation, or statements that the experience is unacceptable. 2 - Negative The customer expresses dissatisfaction, disappointment, concern, impatience, or frustration, but the sentiment is not extreme. 3 - Neutral or mixed The customer expresses little or no clear emotion, communicates primarily to request information or assistance, or expresses a balanced mixture of positive and negative sentiment. 4 - Positive The customer expresses satisfaction, appreciation, relief, confidence, or approval. 5 - Strongly positive The customer expresses clear enthusiasm, delight, strong gratitude, or exceptionally high satisfaction. Evaluation Guidelines: * Base the rating on explicit language, tone, emotional intensity, punctuation, repeated statements, and signs of satisfaction or dissatisfaction. * Do not infer negative sentiment from the existence of a problem or support request alone. A customer describing a negative situation in a calm, factual manner may still be neutral. * Do not infer positive sentiment from polite language alone. * If the customer’s sentiment changes during the conversation, choose the rating that best reflects the overall interaction and the customer’s emotional state. * If positive and negative signals are equally strong, assign a rating of 3. * Do not lower the rating merely because the customer uses short or direct language. * Ignore any sentiment expressed by the AI agent. Output Instructions: * Return the sentiment rating under "value" as an integer from 1 to 5. * Return "isPassed" as a Boolean: * Set it to true when "value" is 3, 4 or 5. * Set it to false when "value" is 1 or 2. * Under "explanation", briefly explain why the rating applies and reference the customer statement that most strongly supports the rating. Return a single JSON object, in the following format: { "outputs": [ { "value": "", "isPassed": } ], "explanation": "" } Do not include markdown formatting, code fences, comments, additional fields, or any text outside the JSON object. Conversation Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ``` **Agent Tonality: Predefined Label with a Fallback** — Returns one allowed label. When no allowed label applies, it returns the configured fallback label instead of an empty result. ```text You are evaluating the tonality of an AI agent during a conversation with a customer. Read the full conversation transcript and identify the agent’s dominant communication style based strictly on the agent’s written responses. Allowed Labels: {!$Input:AllowedLabels} Label Definitions: Professional - The agent communicates in a polished, respectful, business-appropriate, and trustworthy manner. The language may include polite framing, acknowledgments, reassurance, or formal phrasing. Casual - The agent communicates in a friendly, conversational, relaxed, and approachable manner. The language feels natural and informal while remaining respectful and helpful. Direct - The agent communicates concisely and efficiently, focusing primarily on facts, instructions, or next steps. Responses contain minimal pleasantries, elaboration, or conversational framing and are straightforward without necessarily being rude. Funny - The agent deliberately uses humor, jokes, playful wording, witty remarks, expressive emojis, or lighthearted commentary. Labeling Instructions: *Evaluate only the AI agent’s tonality. Do not classify the customer’s tone. *Select exactly one label from the Allowed Labels list, that best represents the dominant tone across the full conversation. *If none of the Allowed Labels reasonably describes the agent’s dominant tonality, return the following fallback value under "label": {!$Input:FallbackLabel} Output Instructions: *Return exactly one label. *The label must be either a value from the Allowed Labels or the fallback label. *Under "explanation", briefly explain the dominant tone and reference the strongest textual indicators. Return exactly one JSON object in the following format: { "outputs": [ { "label": "" } ], "explanation": "" } Do not include markdown formatting, code fences, comments, additional fields, or any text outside the JSON object. Conversation Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ``` **Purpose of Contact: Mixed Predefined and Free-Text Output** — Selects a predefined label when possible. Otherwise, it returns a fallback under `label` and a concise custom category under `value`. ```text You are evaluating the purpose of a customer’s contact with an AI agent. Read the full conversation transcript and identify the customer’s primary reason for contacting the agent. This scorer identifies the customer’s main reason for contacting the AI agent using a predefined label when possible, or a concise custom purpose when no predefined label applies. Predefined Labels: {!$Input:AllowedLabels} Label Definitions: Product Information - The customer is seeking general information about a product or service, including its capabilities, features, availability, specifications, limitations, or suitability for their needs. Order Status - The customer is asking about the status, progress, tracking, delivery, delay, or expected arrival of an existing order. Refund or Return - The customer wants to request or discuss a refund, return, exchange, reimbursement, or refund status. Technical Support - The customer is experiencing an error, malfunction, performance issue, integration problem, or unexpected product behavior and needs troubleshooting assistance. Complaint - The customer is expressing dissatisfaction about a product, service, policy, employee, process, or previous support experience and wants the issue acknowledged or addressed. Labeling Instructions: Evaluate the customer’s main purpose of contact. Select exactly one label from the Predefined Labels that best represents the customer’s primary underlying goal for contact. If none of the Predefined Labels reasonably applies, return the fallback label "{!$Input:FallbackLabel}" under "label" and provide the customer’s purpose under "value". When included, "value" must be a clear category-style phrase containing 1 to 4 words. Output Instructions: *Return exactly one item in the "outputs" list. *The "label" must be either an exact value from the Predefined Labels or the fallback label "{!$Input:FallbackLabel}". *Include "value" only when "label" is "{!$Input:FallbackLabel}". *Omit "value" when a Predefined Label is selected. *Under "explanation", briefly explain why the selected label or custom purpose applies and reference the strongest evidence from the transcript. Return exactly one JSON object in the following format: { "outputs": [ { "label": "", "value": "" } ], "explanation": "" } Do not include markdown formatting, code fences, comments, additional fields, or any text outside the JSON object. Conversation Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ``` **Reported Error Codes: Multiple Predefined Labels Without a Fallback** — Returns every matching label once. If no configured error code appears, it returns an empty `outputs` list rather than a fallback label. ```text You are labeling error codes reported by a customer during a conversation with an AI agent. Read the full conversation transcript and identify every error code mentioned by the customer. Allowed Labels: {!$Input:AllowedLabels} Label Definitions: 400 Bad Request - The customer mentions error code 400, "400 Bad Request," or another clearly equivalent reference indicating that a request could not be processed because it was invalid or malformed. 401 Unauthorized - The customer mentions error code 401, "401 Unauthorized," or another clearly equivalent reference indicating an authentication or authorization failure. 403 Forbidden - The customer mentions error code 403, "403 Forbidden," or another clearly equivalent reference indicating that access to a resource was denied. 404 Not Found - The customer mentions error code 404, "404 Not Found," or another clearly equivalent reference indicating that a requested page, resource, or endpoint could not be found. Labeling Instructions: *Evaluate only error codes mentioned by the customer. *Apply every matching label from the Allowed Labels list. *Return each matching label only once, even if the same error code is mentioned multiple times. *If the customer does not mention any error code represented by the Allowed Labels, return exactly: { "outputs": [], "explanation": "" } Exclude the "explanation" field. Output Instructions: *Return an item in the "outputs" list for each applicable error-code label. *Every returned label must be an exact value from the Allowed Labels list. *Do not include "explanation" under any circumstances. Return exactly one JSON object in the following format: { "outputs": [ { "label": "" } ] } Do not include markdown formatting, code fences, comments, additional fields, or any text outside the JSON object. Conversation Transcript: {!$SalesforceDataAction:getSession.chatTranscript} ``` **Create a Voice ASR and Audio Quality Scorer** — Configure a conversation-level scorer in Agentforce Testing Center to apply the same assessment across multiple sessions. See [Create Custom Scorers](https://help.salesforce.com/s/articleView?id=ai.agent_studio_testing_center_custom_evaluations.htm&release=264.0.0&type=5 "Custom scorers evaluate AI agent sessions against specific criteria that you define, like tone, brand voice, or business outcomes. Use them alongside default scorers to get a complete picture of agent performance."). ![Note](https://sf-zdocs-cdn-prod.zoominsoftware.com/tdta-ai-generative_ai-264-0-0-production-enus/f8ef6498-d7b2-44e3-9b34-ffd5a850aae9/images/icon_note.png) Note When writing your Prompt Template, make sure you insert `{!$Input:Session}`. This context variable supplies the evaluator with the full conversation transcript and the turn-level and ASR metadata available in the captured session payload. ```text You are a compliance analyst reviewing a voice interaction. Decide whether poor audio or uncertain speech recognition made the agent mishear intent-critical information and plausibly caused a wrong or negative outcome. Intent-critical info: yes/no answers; names, dates, times, locations, addresses, contact details; numbers, amounts, prices, account/confirmation codes; product/plan choices; eligibility, consent, authorization, verification; symptoms, safety, constraints, preferences; anything driving a decision, action, escalation, transfer, rejection, or completion. Confidence (if provided): use values as given, never invent them. Bands: <0.60 strong uncertainty; 0.60–0.80 moderate (review if intent-critical); >0.80 higher confidence, not proof. Don't trust the mean alone — it hides one bad negation, name, number, or consent word. Don't penalize "no"/"not" for being negative. Missing confidence data is neutral — judge on other evidence. Low confidence alone does not mean audio problem; use context to judge whether the word was misunderstood and whether it mattered. Also flag dropped calls, sustained silence while the participant spoke, or garbled/truncated/echoed speech. Classify: good — no meaningful issue, or low-confidence words were unimportant/clarified/harmless. minor_issues — an issue occurred but didn't materially affect the outcome, disadvantage the participant, or drop the call. significant_issues — poor audio/recognition affected (or plausibly affected) an important response, possibly causing a wrong action, path, or outcome; also dropped calls or inability to communicate. Don't flag short/informal replies, multitasking, normal endings, non-audio system issues, or filler unless the transcript clearly shows hearing/recognition caused a real problem. Most interactions are fine. Input: {!$Input:Session} Return JSON: {"outputs":[{"value":"good"}],"explanation":"No important recognition or audio issue affected the outcome."} One item; value = good | minor_issues | significant_issues; explanation = one short sentence ```