Process small to medium datasets in Salesforce and write back results as new or updated records without waiting for a scheduled window. Run near real-time data validations and ad hoc processing from Salesforce objects, context definitions, and CSV files by using on-demand definitions.
Required Editions
| View supported editions. |
On-demand definitions can be run in synchronous (immediate) or asynchronous (batch) mode.
Use Cases for On-Demand Processing
- Immediate Analytics: Run definitions whenever your business requires up-to-date data transformations for reporting.
- Rapid Data Validation: Verify transformation logic and data quality instantly by triggering runs during configuration.
- Ad Hoc Requests: Process one-off data without adjusting complex schedules.
- Small Dataset Processing: Handle limited data volumes efficiently without scheduled batch processing.
A Data Processing Engine (DPE) definition calculates a customer's loyalty score based on their purchase history stored in Salesforce objects. Instead of waiting for the nightly batch to run, a service such as a page load or a flow, calls the definition on-demand to get the customer's loyalty score in near real time.
- Configure a Definition for On-Demand Processing
Select the on-demand processing type when you create a Data Processing Engine definition to enable immediate or asynchronous execution. Setting the processing type ensures that the builder shows only the nodes and data sources supported for on-demand processing. - Run an On-Demand Definition
Run an on-demand Data Processing Engine definition directly from the definition builder, or invoke it through a flow by using invocable action, or the run API for synchronous or asynchronous processing. - Validate On-Demand Definitions with Data Preview
Use Data Preview to inspect and validate node-level data transformations in an on-demand definition before you activate and run it. Data Preview for on-demand definitions follows the same workflow as standard Data Processing Engine Data Preview. - Monitor and Troubleshoot On-Demand Definition Runs
Monitor on-demand runs, verify write-back results, and troubleshoot failed executions. - On-Demand Processing Execution Limits
Salesforce Apex guardrails govern on-demand processing of Data Processing Engine (DPE) definitions and are designed for low to moderate data volumes. Review the processing limits and record caps to determine when to use on-demand mode instead of scheduled batch processing. - Supported Nodes and Data Sources for On-Demand Processing
Review the data sources and transformation nodes compatible with on-demand processing. On-demand mode is optimized for targeted transformations on Salesforce data. Verify that the transformation logic is compatible before you activate a definition. - Integrate Data Processing Engine with Context Service
Integrate the Data Processing Engine (DPE) definitions with Context Service to read data from context definitions, transform it, and write it back to a specified JSON structure for downstream processes.