---
schema_version: "1"
id: ca21dfd165775f5ed79f30898b87fa964447bf2bb14629b50950b42e42edac78
title: Create Custom Scorers
collection: admin
version: current
locale: en-us
portal_url: https://docs.salesforce.com/admin/current/en-us/ai/264-0-0/agent_studio_testing_center_custom_evaluations.md
canonical_url: https://help.salesforce.com/s/articleView?id=ai.agent_studio_testing_center_custom_evaluations.htm&release=264.0.0&type=5
source_site: help.salesforce.com
content_type: documentation
content_hash: 3da47a839edbed796c49627e97ef3e734db1feabaea85496ed57c0e6b0ec8274
description: Custom scorers evaluate AI agent sessions against specific criteria that you define, like tone, brand voice, or business outcomes. Use them alongside default scorers to get a complete picture of agent performance.
products:
  - Agentforce
  - Cross Cloud
release: "264"
---
# Create Custom Scorers

Custom scorers evaluate AI agent sessions against specific criteria that you define, like tone, brand voice, or business outcomes. Use them alongside default scorers to get a complete picture of agent performance.

### Required Editions

<table class="slds-table slds-table_bordered slds-max-medium-table_stacked slds-m-bottom_small ">
        <colgroup>
          <col>
          <col>
        </colgroup>
        <tbody>
          <tr class="">
            <td class="slds-cell-wrap" style="vertical-align:top;"><a name="agent_studio_testing_center_custom_evaluations__image_ewv_dvs_c3c"><!-- --></a>
              <div><img id="agent_studio_testing_center_custom_evaluations__image_ewv_dvs_c3c" src="https://sf-zdocs-cdn-prod.zoominsoftware.com/tdta-ai-generative_ai-264-0-0-production-enus/f8ef6498-d7b2-44e3-9b34-ffd5a850aae9/generative_ai/resources/images/checkmark.png" alt="green checkmark"></div>
              <p style="margin-bottom:0;"><strong>This article applies to:</strong></p>
            </td>

            <td class="slds-cell-wrap" style="vertical-align:top;">New Testing Center in Agentforce Studio (Beta)</td>

          </tr>
          <tr class="">
            <td class="slds-cell-wrap" style="vertical-align:top;"><a name="agent_studio_testing_center_custom_evaluations__image_qxv_cvs_c3c"><!-- --></a>
              <div><img id="agent_studio_testing_center_custom_evaluations__image_qxv_cvs_c3c" src="https://sf-zdocs-cdn-prod.zoominsoftware.com/tdta-ai-generative_ai-264-0-0-production-enus/f8ef6498-d7b2-44e3-9b34-ffd5a850aae9/generative_ai/resources/images/crossmark.png" alt="red crossmark"></div>
              <p style="margin-bottom:0;"><strong>This article doesn’t apply to:</strong></p>
            </td>

            <td class="slds-cell-wrap" style="vertical-align:top;">Legacy Agentforce Testing Center in Setup</td>

          </tr>
        </tbody>
      </table>

<table class="slds-table slds-table_bordered slds-max-medium-table_stacked slds-m-bottom_small edition"><colgroup><col></colgroup><tbody><tr class=""><td class="slds-cell-wrap" style="vertical-align:top;">Available in: Lightning Experience</td></tr><tr class=""><td class="slds-cell-wrap" style="vertical-align:top;">Available in: <strong>Enterprise</strong>, <strong>Performance</strong>, <strong>Unlimited</strong>, and <strong>Developer</strong> Editions. <a href="https://help.salesforce.com/s/articleView?id=ai.agent_setup_explore_types.htm&amp;release=264.0.0&amp;type=5" title="Learn about agent types and default templates for specific clouds and common use cases.">Required add-on licenses vary by agent type.</a></td></tr></tbody></table>

<table class="slds-table slds-table_bordered slds-max-medium-table_stacked slds-m-bottom_small permission">
        <colgroup>
          <col style="width:50%">
          <col style="width:50%">
        </colgroup>
        <thead class="slds-text-title_caps ">
          <tr class="">
            <th scope="colgroup" class="slds-cell-wrap" colspan="2" style="text-align:center;vertical-align:top;" id="d77479e123">User Permissions Needed</th>
          </tr>
        </thead>
        <tbody>
          <tr class="">
            <td data-label="User Permissions Needed" class="slds-cell-wrap" style="vertical-align:top;" headers="d77479e123 ">To create tests in the Testing Center:</td>

            <td class="slds-cell-wrap" style="vertical-align:top;" headers="d77479e123 ">
              <p>Manage Agentforce Grids</p>
              <p>AND</p>
              <p style="margin-bottom:0;">Manage Agentforce Testing</p>
            </td>

          </tr>
        </tbody>
      </table>

Along with default scorers, custom scorers let you define specific criteria to assess your agent's effectiveness. With custom scorers you can evaluate resolution quality, policy adherence, sentiment, and any business-specific outcome your team cares about. Or, use custom scorers when you expect a structured output like a JSON.

Custom scorers use an LLM-as-judge: you define the evaluation criteria in a prompt template, and the AI model uses it to assess each session and return a result based on the output type you configure.

Custom scorers return results in two ways based on the output type you select.

- Labels. Classifies each session using a predefined set of values. Use this pattern for classification where outputs must stay within approved values.
- Free-form. Returns output based on your prompt, like text, a number, or a URL. Use this pattern for richer insights like summaries, or sentiment signals.

Once you've decided which output type fits your use case, create a custom scorer in your test suite.

1. To add or review your scorers from the test suite, click **Select Scorers**.
2. Click **Add** and select **LLM Judge**.
    
    **What is an LLM judge?**
    
    An LLM judge (or LLM-as-judge) is when one large language model (LLM) evaluates and scores the outputs of another. A judge LLM receives a prompt that defines the task and outlines the scoring criteria like factual accuracy, relevance, coherence, and faithfulness to source. With these resources and guidelines, the LLM judge determines the expected response and compares it to the agent response. Based on this comparison, the judge generates scores, rankings, or written feedback. The LLM as judge model serves as a scalable, automated, and objective evaluation tool for tasks like scoring summaries or ranking responses. We’ve carefully designed our LLM-as-judge prompts to give you the most accurate and useful test results.
3. Give the custom scorer a **Name** and **API Name**, and select a **Response Data Type**.
    - Select **Labels** to classify each session with one value from a predefined set you define. Labels constrain the AI model to return only values from your predefined set. You reference these values in your prompt template via merge fields, and Agentforce injects them at runtime so the AI model knows its valid options when evaluating each session.
    - Select **Boolean**, **DateTime**, **Number**, **Text**, or **URL** to return free-form output based on your prompt.
4. If you selected **Labels**, configure the label fields.
    1. In **Allowed Labels**, enter the label values the AI model may apply (for example, `TRUE`, `FALSE`).
        
        These values are injected into your prompt via merge fields. Your prompt instructs the AI model to return exactly one of them as the scorer result for each session.
    2. In **Fallback Label**, enter the label to apply when the AI model can't determine a result or when prompt execution fails (for example, `UNSURE`).
5. Create the evaluation prompt in the **Prompt Template Composer**.
    1. Select the AI model to serve as the judge.
    2. Write the evaluation prompt. For Labels scorers, include merge fields to supply the conversation transcript and your label values to the AI model at runtime.
        
        Your prompt should define the evaluation criteria and describe when each label applies. Merge fields supply runtime data the AI model can't access otherwise.
        
        ![Important](https://sf-zdocs-cdn-prod.zoominsoftware.com/tdta-ai-generative_ai-264-0-0-production-enus/f8ef6498-d7b2-44e3-9b34-ffd5a850aae9/images/icon_important.png)
        
        Important Session-level and turn-level tests use different data models. Make sure the merge fields in your prompt match the test scope you plan to activate the scorer for.
        
        `{!$Input:AllowedLabels}`
        
        Tells the AI model which label values it can return. Without this, the model may generate values outside your defined set.
        
        `{!$Input:FallbackLabel}`
        
        Tells the AI model what to return when it can't determine a result.
        
        `{!$SalesforceDataAction:getSession.chatTranscript}`
        
        Supplies the full conversation transcript. Use for session-level (conversational) scoring.
        
        `{!$SalesforceDataAction:getLastInteraction.userUtterance}`
        
        Supplies the user's last message. Use for turn-level (intent) scoring.
        
        `{!$SalesforceDataAction:getLastInteraction.agentResponse}`
        
        Supplies the agent's last response. Use for turn-level (intent) scoring.
        
        For complete API reference, see [Custom Scorers API Reference](https://developer.salesforce.com/docs/ai/agentforce/guide/testing-api-custom-scorers.html).
    3. Optionally, save multiple template versions.
    4. Set the threshold score for your passing criteria.
6. Save your scorer.

After you save your custom scorer, it's automatically selected for your test suite. Once Agentforce finishes generating tests, they appear under Test Cases. Depending on the size and complexity of the request, this process can take anywhere from a few minutes to several hours. You may need to refresh the testing suite to see your test cases.

When the test cases appear, click **Run Test**. Agentforce begins running the chosen scorers on the test cases.