Data Processing Engine Key Terms

Browse through this collection of terms, key objects, and key concepts. This collection is designed to give admins, designers, and developers a clear and consistent understanding of concepts to navigate the Data Processing Engine landscape.

Required Editions

View supported editions.

Important Where possible, we changed noninclusive terms to align with our company value of Equality. We maintained certain terms to avoid any effect on customer implementations.

Term	Description
Activation	Marking a Data Processing Engine definition as ready to run.
Batch Job	A scheduled Data Processing Engine definition that processes a large volume of data or performs a series of tasks without requiring user interaction. The jobs are typically run in the background and can include tasks such as data transformation, data loading, and report generation.
Batch Job Part	A smaller, manageable segment of a larger batch job.
Calculated insight object (CIO)	A data model object that's created when a calculated insight is processed. You can view CIOs in the Data Explorer. See Data Objects in Data 360.
Composite Writeback	The method of grouping related records from different writeback nodes and writing them back in a single operation, at run time. This method makes sure that the data remains consistent and prevents partial updates that can lead to data corruption.
CRM Analytics	A data analysis and visualization service for Salesforce CRM that provides actionable insights and AI-driven analytics in the user's workflow. See CRM Analytics.
Data 360	A product used to organize and unify data across Salesforce and other external data sources. See About Salesforce Data 360.
Data source	The data that needs to be transformed. Data sources can be standard or custom objects, Data 360 objects, or Analytics datasets. Depending on the Data Processing Engine runtime that you select when you create a definition, you can select the source type for data source node. You can select these data source types. Salesforce Objects: Standard objects such as Account, Contact, and Lead and custom objects that you can create to meet specific business needs. Analytics Datasets: Data from CRM Analytics datasets. External Data Sources: Data that can be sourced from external databases or systems by using APIs, CSV files, or third-party integrations. Data 360 Objects: Data Model Objects (DMO), Data Lake Objects (DLO), and Calculated Insights Objects (CIO).
Data space	A governance feature in Data 360 that helps administrators create logical partitions to organize data. These partitions help segregate data, metadata, and processes into categories such as brand, region, and department. See Data Spaces.
Data stream	The connection and associated data ingested in Data 360. Data streams enable the import of data from multiple sources, such as APIs, files, and emails.
Data lake object (DLO)	Storage containers within the data lake for the data ingested into all data streams within Data 360. See Data Objects in Data 360.
Data model object (DMO)	A grouping of data created from data streams, insights, and other sources. DMOs can be standard or custom, based on business need. See Data Objects in Data 360.
Definition	Specifies what data must be processed, how the data must be processed, and where the processed results must be stored. A Data Processing Engine definition must include the source data, the processing logic, and the destination where the results must be saved.
Failed records	A record that wasn't processed properly during a batch job. The failure can occur due to various reasons such as field validation errors, data type mismatch, and missing required fields.
Failed records file	The file that captures the records that weren't successfully processed during a batch job. Use this file to investigate and troubleshoot the errors or issues in data entries.
Field mapping	The process of defining how the data fields from a source node correspond to the fields in a target object.
Nodes	These are the building blocks of a definition used to configure data sources, data transformation logic, and writeback targets. Data source nodes: Help you select the sources of data that the definition must transform. Transformation nodes: Help you configure a logical set of transformations steps that decide how data is filtered, grouped, aggregated, and used to calculate formulas. Writeback nodes: Help you decide that the target locations where the transformed results write back to.
Output records	The processed records that are written to the target object when a Data Processing Engine definition is run.
Recipe	Set of instructions that tells Data Processing Engine how to process and transform data from various Salesforce objects and potentially external sources.
Related objects	The objects that are related to each other via a lookup, master-detail, or many-to-many relationships.
Related fields	The fields that link the records of two or more related objects.
Related field mapping	Mapping fields between the related objects in an org. Use related mapping to: Map fields between parent and child objects such as Accounts and Contacts in a parent-child relationship. Map fields that reference other objects through lookup fields. Map fields between Salesforce objects and external data sources to establish relationships between data in different orgs. Map fields between custom objects and the fields that you have defined in your business. For example, if you're importing data that includes both Account and Contact information, related field mapping makes sure that each Contact is associated with the appropriate Account. Map an Account ID field in the source data to the Account lookup field in the Contact object in Salesforce. Correctly set up related field mappings to make sure that the integrity of the data relationships is maintained, which is crucial for accurate data reporting, querying, and overall data management.
Runtime	The platform where data processing tasks are run. A runtime defines the set of operations and transformations that can be performed on the data, and the infrastructure used to carry out these tasks.
Run	The process of running a Data Processing Engine definition in the CRMA runtime or Data 360 runtime.
Scheduling	The process of defining when and how often a batch job must be executed. Scheduling makes sure that data processing tasks are performed at appropriate times, whether on a recurring basis or in response to specific events. Scheduling makes sure that data processing tasks are performed consistently, efficiently, and in a timely manner.
Writeback	The process of sending or writing processed or transformed data as new or updated records in the target object.