Connect API Resources for Testing API

The Connect API endpoints for Testing API execute test cases, poll for results, and programmatically retrieve detailed test results.

Start Test

Start a test asynchronously based on the provided AiEvaluationDefinition name or ID. This endpoint schedules the test and returns an identifier to track its progress. OAuth 2.0 with a connected app is required. See the Connect REST API Quick Start.

Resource

/services/data/v63.0/einstein/ai-evaluations/runs

HTTP Methods

POST

Request

Headers

{
"Content-Type": "application/json",
"Authorization": "Bearer <access_token>"
}

Parameters

{
"aiEvaluationDefinitionName": "<string>"
}

The API enforces that exactly one of the two parameters is provided. If no parameters or multiple parameters are provided, the API returns a 400 Bad Request response code with an empty error message.

Response

HTTP response code: 200

{
  "runId": "<string>",
  "status": "<string>"
}

Get Test Status

Retrieves the operational status of a specified test. We designed this endpoint to poll to monitor the progress of a test.

Resource

/einstein/ai-evaluations/runs/:runId

HTTP Methods

GET

Request

Headers

{
"Authorization": "Bearer <token>"
}

Parameters

Response

{
  "status": "<enum>",
  "startTime": "<ISO 8601 timestamp>",
  "endTime": "<ISO 8601 timestamp>",
  "errorMessage": "<string>"
}

Get Test Results

Retrieve a detailed report of a test, including information on each test case and the results of all predetermined expectations.

Resource

/einstein/ai-evaluations/runs/:runId/results

HTTP Methods

GET

Request

Headers

{
"Authorization": "Bearer <token>"
}

Parameters

Response

{
  "status": "<ENUM(“NEW”, “IN_PROGRESS”, “COMPLETED”, “ERROR”)>",
  "subjectName": "<string>",
  "startTime": "<ISO 8601 timestamp|null>",
  "endTime": "<ISO 8601 timestamp|null>",
  "errorMessage": "<string|null>",
  "testCases": [
    {
      "status": "<ENUM(“NEW”, “IN_PROGRESS”, “COMPLETED”, “ERROR”)>",
      "testNumber": "<integer>",
      "startTime": "<ISO 8601 timestamp|null>",
      "endTime": "<ISO 8601 timestamp|null>",
      "errorMessage": "<string|null>",
      "inputs": {
        "utterance": "<string>"
      },
      "generatedData": {
        "actionsSequence": "<string[]|null>",
        "outcome": "<string|null>",
        "topic": "<string|null>"
      },
      "testResults": [
        {
          "name": "<string>",
          "actualValue": "<string|null>",
          "expectedValue": "<string|null>",
          "metricScore": "<ENUM('PASS','FAILED','HIGH','LOW','UNCERTAIN')|null>",
          "metricLabel": "<string|null>",
          "metricExplainability": "<string|null>",
          "status": "<ENUM(“NEW”, “IN_PROGRESS”, “COMPLETED”, “ERROR”)>",
          "startTime": "<ISO 8601 timestamp|null>",
          "endTime": "<ISO 8601 timestamp|null>",
          "errorCode": "<string|null>",
          "errorMessage": "<string|null>"
        }
      ]
    }
  ]
}

Response Object

testCase Object

inputs Object

generatedData Object

testResult Object