| Digital Wallet Card Consumption Card | Usage Type | Usage Type Description | Notes |
|---|---|---|---|
| Data Services | Batch Data Pipeline (External Data Pipeline) | Usage is calculated based on the number of rows batch data processed by Data 360 data streams across all connectors, with the exception of structured data ingested via the Internal Data Pipeline. |
Usage is calculated based on the number of records that are ingested in batch mode by data streams. Unstructured data may be ingested through any Data 360 connector. For unstructured data files that are only referenced - and not ingested - from an external blob store, such as Amazon S3, there is no ingestion cost. |
| Data Services | Unstructured Data Processed |
Usage is calculated based on the amount of unstructured data that is processed without using AI assisted content processing and extraction. For example, if the search index processes 100 PDF documents that are 1 MB each, usage is calculated as 100 MB. If the search index processes five audio/video files that are on average 100MB each, usage is calculated as 500MB. In Data 360, unstructured data may be chunked and vectorized using an embedding model. Usage is computed only once across both these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB. If you create a search index that uses a data model object (DMO) as a data source and select Text data type fields from the DMO, all strings from those fields are indexed and the sum total of all bytes processed is reported as Unstructured Data Processed. |
In Data 360, unstructured data may be chunked and vectorized using an embedding model. For audio and video files, a text transcript is created before the files are chunked. Usage is computed only once across all these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB. If a video file of 1GB is transcribed, chunked, and vectorized, usage is computed as 1GB. A DMO and all its file attachments are treated as a single unit for processing purposes. For example, if Data 360 processes incremental changes either to the fields on a source DMO or to a file attachment on that DMO, all file attachments are reindexed. The cost of creating a search index remains the same for vector search index and hybrid search index. |
| Flex Credits | Data 360 Unstructured Processing |
Usage is calculated based on the amount of unstructured data that is processed without using AI assisted content processing and extraction. For example, if the search index processes 100 PDF documents that are 1 MB each, usage is calculated as 100 MB. If the search index processes five audio/video files that are on average 100MB each, usage is calculated as 500MB. In Data Cloud, unstructured data may be chunked and vectorized using an embedding model. Usage is computed only once across both these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB.If you create a search index that uses a data model object (DMO) as a data source and select Text data type fields from the DMO, all strings from those fields are indexed and the sum total of all bytes processed is reported as Unstructured Data Processed. |
In Data 360, unstructured data may be chunked and vectorized using an embedding model. For audio and video files, a text transcript is created before the files are chunked. Usage is computed only once across all these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB. If a video file of 1GB is transcribed, chunked, and vectorized, usage is computed as 1GB. A DMO and all its file attachments are treated as a single unit for processing purposes. For example, if Data 360 processes incremental changes either to the fields on a source DMO or to a file attachment on that DMO, all file attachments are reindexed. The cost of creating a search index remains the same for vector search index and hybrid search index. |
| Data Services | Intelligent Processing | Usage is calculated based on the amount of unstructured data that is processed using AI-assisted features such as LLM-based parsing, LLM-based visual data preprocessing, image processing, Enriched Indexing, and Intelligent Context. |
When LLM-based parsing is used, entire documents are sent to the LLM for processing, and size of all documents is reported against "Intelligent Processing” usage type. When LLM-based visual data preprocessing is used, only content that contains visual elements or tables is sent to LLM for processing. Size of all documents from which any content is sent to the LLM for processing is reported against "Intelligent Processing” usage type. The sizes of all other documents from which no content is sent to the LLM for processing are reported against “Unstructured Data Processed” usage type. The same guideline applies for documents that are uploaded and indexed in Intelligent Context. With LLM-based image processing, the entire document is sent to the LLM for processing, and size of the document is reported against "Intelligent Processing” usage type. When Enriched Indexing is enabled, calls to the LLM to generate enriched chunks are reported against "Intelligent Processing” usage type. |
| Flex Credits | Data 360 Intelligent Processing | Usage is calculated based on the amount of unstructured data that is processed using AI-assisted features such as LLM-based parsing, LLM-based visual data preprocessing, image processing, Enriched Indexing, and Intelligent Context. |
When LLM-based parsing is used, entire documents are sent to the LLM for processing, and size of all documents is reported against "Intelligent Processing” usage type. When LLM-based visual data preprocessing is used, only content that contains visual elements or tables is sent to LLM for processing. Size of all documents from which any content is sent to the LLM for processing is reported against " Data 360 Intelligent Processing” usage type. The sizes of all other documents from which no content is sent to the LLM for processing are reported against “ Data 360 Unstructured Data Processed” usage type. The same guideline applies for documents that are uploaded and indexed in Intelligent Context. With LLM-based image processing, the entire document is sent to the LLM for processing, and size of the document is reported against " Data 360 Intelligent Processing” usage type. When Enriched Indexing is enabled, calls to the LLM to generate enriched chunks are reported against "Data 360 Intelligent Processing” usage type. |
| Data Services | Data Queries |
Usage is calculated based on the number of records processed. The count of records processed depends on the structure of a query as well as other related factors such as the total number of records in the objects being queried. |
For vector search queries against unstructured data, the number of vectors in the search index are counted. For hybrid search queries against unstructured data, the number of vectors and keyword records in the search index are counted. In a typical search index, the number of keyword records is the same as the number of vectors. |
| Flex Credits | Data 360 Queries |
Usage is calculated based on the number of records processed. The count of records processed depends on the structure of a query as well as other related factors such as the total number of records in the objects being queried. When an object is queried in the context of a data space, the number of rows processed is based on the total number of processed rows in the source object, not the number of rows pertaining to the particular data space |
For vector search queries against unstructured data, the number of vectors in the search index are counted. For hybrid search queries against unstructured data, the number of vectors and keyword records in the search index are counted. In a typical search index, the number of keyword records is the same as the number of vectors. |
| Digital Wallet Consumption Card | Usage Type | Usage Type Description | Notes |
|---|---|---|---|
| Data Storage | Storage Beyond Allocation | Usage is calculated based on the amount of storage used above the amount allocated. |
Every file ingested and table created, including unstructured data lake objects (UDLO), unstructured data model objects (UDMO), CDMO, or index data model objects (index DMO), count toward Data 360 data storage, including the following.
|