Regression testing compares a new Tableau Agent answer with the verified query you saved. A match stays Verified. A mismatch becomes Regression.
Required Editions
| Available in: Tableau Next |
| Available in: Enterprise, Performance, and Unlimited Editions |
What a Regression Test Compares
Regression testing checks whether Tableau Agent still returns the answer you saved for a verified query.
A verified query is a question you classified as correct, together with the query that produced that answer. Tableau Agent uses verified queries as guidance so similar questions return a consistent, accurate answer.
Those answers depend on the semantic model: the names, descriptions, fields, and relationships Tableau Agent uses to understand the question. When the semantic model changes, the same question can produce a different query. The question text stays the same. The answer can differ from the one you verified.
Regression testing runs the verified question again. Tableau Agent returns a new answer, and Q&A Calibration compares that answer with the saved query. The saved query is the expected result. The new answer is the actual result. The test compares query logic. The SQL text can differ.
Pass and Fail Results
A pass means the saved query and the new query use the same logic. The SQL text can differ. The classification stays Verified, and Tableau Agent continues to use the verified query as guidance.
A fail means the new query logic differs from the saved query. The classification changes to Regression, and Tableau Agent stops using the verified query as guidance. The saved query is no longer executable. Revise the verified query so that it matches the current semantic model. In most cases, the semantic model change is correct and the saved query is out of date.
Review Analysis opens Review Differences. Q&A Calibration shows the saved query on the left and labels it Verified Expected Query. The new answer appears on the right. The note under the queries is Response Match Analysis for a pass and for a fail.
To run this check on one question, see Test a Verified Query for Regression in Q&A Calibration.
- Test a Verified Query for Regression in Q&A Calibration
Open a verified query in Q&A Calibration. Tableau Agent runs a regression test that compares the new answer with the query you saved. Retest after a semantic model change.