Run Vector Searches Using Query API

You can use Data 360 SQL to write custom vector search statements. With the Data 360 Query API you can use these SQL statements in your custom applications.

Basic Vector Search

This syntax performs a vector search across an index data model object (DMO) to find the defined string.

    
SELECT * 
FROM vector_search(TABLE(<Your_Vector_Embedding DMO>), 
  '<Search_String>', 
  '<PreFilteringColumn><Operator>"<Value>"', 
  '<k=topK results>'
)

Where:

<Your_Vector_Embedding>

The API name of the index DMO.

Search_String

The search string for the vector search.

PrefilteringColumn

A boolean expression that filters the scalar fields of the primary key field to limit the search. For more information about pre-filtering, see Vector Search Query Expressions and Pre-Filtering.

k=topK_results

The number of the most similar results to return.

Vector Search With Index and Chunk DMOs

This syntax performs a vector search across an index DMO and a chunk DMO.

SELECT
  v.score__c AS Score__c,
  c.chunk__c AS Chunk__c
FROM
  vector_search(
    TABLE(<Your_Vector_Embedding_DMO>),
    '<Search_String>',
    'language = \'English\' AND status = \'active\'',
    10
    ) v
JOIN
  <Your Chunk DMO> c ON v.RecordId__c = c.RecordId__c
  AND c.KQ_RecordId__c IS NOT DISTINCT FROM v.KQ_RecordId__c
ORDER BY 1
LIMIT 10
      

Where:

Score__c

The similarity score of each vector to the input search string.

Chunk__c

The chunk that was used to generate the vector embedding.

language=\'English\' AND status=\'active\'

An example of prefiltering with two filters. These filters are combined using the logical AND operator.

10

Returns the top 10 results.

<Your Chunk DMO>

The API name of the chunk DMO.

RecordId__c and KQ_RecordId__c

The join statement matches on both the RecordID and the key qualified record ID while avoiding NULL values. When joining DMOs and DLOs, use key qualifiers to avoid ID overlap.

Vector Search with DMOs and Unstructured Data Model Objects (UDMO)

This example performs a vector search, retrieves the relevant text chunks and their scores, provides the file paths of the source documents for the chunks, and returns the top 10 results.

SELECT
  v.score__c AS Score__c,
  c.chunk__c AS Chunk__c,
  u.FilePath__c AS FilePath__c
FROM
  vector_search(TABLE(<Your Vector_Embedding DMO>), '<Search String>', '', 10) v
JOIN
  <Your_Chunk_DMO> c ON v.RecordId__c = c.RecordId__c
  AND c.KQ_RecordId__c IS NOT DISTINCT FROM v.KQ_RecordId__c
JOIN
  <Your_UDMO> u ON c.SourceRecordId__c = u.FilePath__c
ORDER BY 1 DESC
LIMIT 10;
      

Where:

Your_UDMO

The API name of the UDMO used to create the chunks.

FilePath__c

The relative path of each file that was used to create the chunks and vector embeddings.