--- schema_version: "1" id: a94ccf99f0cac9eff7e1b0ca72f9421368b3facd0376fa66eea5fc18280df3e6 title: agent test run-eval (Beta) collection: developer version: current locale: en-us portal_url: https://docs.salesforce.com/developer/current/en-us/salesforcedx/sfclireference/guides/agent/cli_reference_agent_test_run-eval.md canonical_url: https://developer.salesforce.com/docs/platform/salesforce-cli-reference/guide/cli_reference_agent_test_run-eval.html source_site: developer.salesforce.com content_type: documentation content_hash: 76156e4e6d09ee9e8a9b9f33e2a6947640b503739cd51c91682842eda463b878 products: - Salesforce CLI Command Reference - salesforcedx --- # agent test run-eval (Beta) Run rich evaluation tests against an Agentforce agent. **note:** This feature is a Beta Service. Customers may opt to try such Beta Service in its sole discretion. Any use of the Beta Service is subject to the applicable Beta Services Terms provided at [Agreements and Terms](https://www.salesforce.com/company/legal/agreements/). ## Description for agent test run-eval Specify the tests you want to run with one of these inputs to the --spec flag: When you provide a YAML test spec, this command automatically translates test cases into internal state-based evaluation framework calls and infers the agent name from the test spec's `subjectName` field. As a result, you can use the same test spec with both the `agent test run` and `agent test run-eval` commands. YAML test specs also support context variables, which allow you to inject contextual data (such as CaseId or RoutableId) into agent sessions for testing with different contexts. When you provide a JSON payload, it's sent directly to the evaluation framework with optional normalization. The normalizer auto-corrects common field name mistakes, converts shorthand references to JSONPath, and injects defaults. Use `--no-normalize` to disable this auto-normalization. JSON payloads can also include context_variables on agent.create_session steps for the same contextual testing capabilities as when you use a YAML test spec. This command supports more than 8 evaluator types, including subagent routing assertions, action invocation checks, string/numeric assertions, semantic similarity scoring, and LLM-based quality ratings. ## Flags | Flag Name (Long) | Flag Name (Short) | Description | |------------------|-------------------|-------------| | ‑‑api‑name | ‑n | **Type:** Value

Agent API name (also called DeveloperName) used to resolve agent_id and agent_version_id. Auto-inferred from the YAML spec's subjectName. | | ‑‑api‑version | N/A | **Type:** Value

Override the api version used for api requests made by this command | | ‑‑batch‑size | N/A | **Type:** Value
**Default value:** `5`

Number of tests per API request (max 5). | | ‑‑flags‑dir | N/A | **Type:** Value

Import flag values from a directory. | | ‑‑json | N/A | **Type:** Boolean

Format output as json. | | ‑‑no‑normalize | N/A | **Type:** Boolean

Disable auto-normalization of field names and shorthand references. | | ‑‑result‑format | N/A | **Type:** Value
**Valid Values:** `json`, `human`, `junit`, `tap`
**Default value:** `human`

Format of the agent test run results. | | ‑‑spec | ‑s | **Type:** Value
**Required**

Path to test spec file (YAML or JSON). Supports reading from stdin when piping content. | | ‑‑target‑org | ‑o | **Type:** Value
**Required**

Username or alias of the target org. Not required if the `target-org` configuration variable is already set. | ## Examples for agent test run-eval Run tests using a YAML test spec on the org with alias "my-org": ```shell sf agent test run-eval --spec specs/my-agent-testSpec.yaml --target-org my-org ``` Run tests using a YAML spec with explicit agent name override; use your default org: ```shell sf agent test run-eval --spec specs/my-agent-testSpec.yaml --api-name My_Agent ``` Run tests using a JSON payload: ```shell sf agent test run-eval --spec specs/eval-payload.json --target-org my-org ``` Run tests and output results in JUnit format; useful for continuous integration and deployment (CI/CD): ```shell sf agent test run-eval --spec specs/my-agent-testSpec.yaml --target-org my-org --result-format junit ``` Run tests with contextVariables to inject contextual data into agent sessions (add contextVariables to test cases in your YAML spec): ```shell sf agent test run-eval --spec specs/agent-with-context.yaml --target-org my-org ``` Pipe JSON payload from stdin (--spec flag is automatically populated from stdin): ```shell echo '{"tests":[...]}' | sf agent test run-eval --spec --target-org my-org ```