---
schema_version: "1"
id: af6dc1329fc7118f1a8af03f931e0d546d2c31134dd4210236451d03254d2571
title: Batch Component Reference
collection: mulesoft
version: current
locale: en-us
portal_url: https://docs.salesforce.com/mulesoft/current/en-us/mule-runtime/4.6/batch-reference.md
canonical_url: https://docs.mulesoft.com/mule-runtime/4.6/batch-reference
source_site: docs.mulesoft.com
content_type: documentation
content_hash: 2cb53e450505b5dbd5ef884d1618a7439a0bc69dc77dbbe56e339bd47a8367b2
description: MuleSoft Documentation Site
release: "4.6"
latest: false
---
---
product: Mule Runtime
version: 4.6
---

# Batch Component Reference

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Mule batch components manage bulk processing of records with minimal adjustments to default
component settings. They are highly configurable to meet specific processing and performance
requirements.

A Batch Job component must contain at least one Batch Step component. Optionally, each Batch Step
component can contain a single Batch Aggregator component. The Mule flow in the following figure
shows batch components.

![Batch Processing Components in a Mule Flow within Studio](/mulesoft/current/en-us/mule-runtime/4.6/_images/batch-flow-generic-example.png)

The Batch Job component splits the payload of the Mule message into records. Within the Batch Step
and Batch Aggregator components, these records are available with the keyword `payload`, and Mule
variables are also accessible through the `vars` keyword. However, Mule attributes are not
accessible from within these components. Both components return `null` on `attributes`.

This reference assumes that you are familiar with the [batch processing phases](https://docs.mulesoft.com/mule-runtime/4.6/batch-phases) of a
batch job instance.

## Batch Job Component (<batch:job />)

This component prepares its input for processing and outputs a report with results of that
processing. Preparation takes place during the [Load and Dispatch](https://docs.mulesoft.com/mule-runtime/4.6/batch-phases#phase_load_dispatch)
phase of a batch job instance, and the report is available in the
[On Complete](https://docs.mulesoft.com/mule-runtime/4.6/batch-phases#phase_on_complete) phase.

In Studio, you can set these properties in the component’s [General](#ref_batch_job_general_fields)
and [History](#ref_batch_job_history_fields) tabs.

### Batch Job Properties

The Batch Job component provides a number of configurable properties that determine how batch
processing takes place. It also provides a way of storing the input payload in a target variable for
use by subsequent components in the flow, which can be useful because the Batch Job component
consumes the message payload.

Anypoint Studio provides the following fields in the **General** tab:

![Batch Job fields in Studio](/mulesoft/current/en-us/mule-runtime/4.6/_images/batch-job-fields-studio.png)

The following table describes each field and includes the XML attribute for the field.

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Field Name</th><th>XML</th><th>Description</th></tr></thead><tbody><tr><td><div><div><p>Name</p></div></div></td><td><div><div><p><code>name</code></p></div></div></td><td><div><div><p>Configurable name for a Batch Job component. The default name appends <code>Batch_Job</code> to the name of the application.</p></div></div></td></tr><tr><td><div><div><p>Max Failed Records</p></div></div></td><td><div><div><p><code>maxFailedRecords</code></p></div></div></td><td><div><div><p>Maximum number of records that can fail within a batch job instance before the execution of that instance stops. The default is <code>0</code>. Use <code>-1</code> for no limit. This setting takes precedence over any accept policy (<code>acceptPolicy</code>) or expression (<code>acceptExpression</code>) filter on a <a href="#ref_batch_step_component">Batch Step component</a> within the Batch Job component. When a batch job instance exceeds the <code>maxFailedRecords</code> value, the Batch Job component stops processing of the batch job instance in which the failure occurs, assigns the instance the status <code>FAILED_PROCESS_RECORDS</code>, and reports the number of failed records in the instance. When a batch job instance reaches this threshold, the console prints a message such as this one (edited for readability):</p></div><div><div><pre>INFO  ... DefaultBatchEngine:
 instance '6d85c380-f332-11ec-bb5f-147ddaaf4f97' of job 'Batch_Job_in_Mule_Flow'
 has reached the max allowed number of failed records. Record will be added to
 failed list and the instance will be removed from execution pool.
...
INFO  ... DefaultBatchEngine:
 instance 6d85c380-f332-11ec-bb5f-147ddaaf4f97 of job Batch_Job_in_Mule_Flow
 has been stopped. Instance status is FAILED_PROCESS_RECORDS</pre></div></div><div><p>Note that parallel processing within a batch job instance makes it possible for the number of errors to exceed the configured maximum because it is possible for multiple records fail at the same time.</p></div></div></td></tr><tr><td><div><div><p>Scheduling Strategy</p></div></div></td><td><div><div><p><code>schedulingStrategy</code></p></div></div></td><td><div><div><p>When a Batch Job component triggers repeatedly, more than one batch job instance can be ready to execute at the same time. The instances can run sequentially based on their creation timestamp (the default) or according to a round-robin algorithm that attempts to maximize the use of available resources. This setting is appropriate only for jobs that have no side effects or dependencies on other records because the order of processing is not guaranteed.</p></div><div><ul><li><p><code>ORDERED_SQUENTIAL</code> (default): With this setting, batch job instances execute one at a time in the order established by their creation timestamp. An instance created earlier executes before an instance created at a later time and date.</p></li><li><p><code>ROUND_ROBIN</code>: With this setting, all batch job instances execute based on a round-robin algorithm that assigns available resources. Order of execution is not predetermined. Batch instances can execute in any order. This option is viable only if you are certain that no execution of an instance is capable of producing a side effect on an execution of another instance. Best practices:</p><div><ul><li><p>To avoid potential updates to the same record in concurrently running batch job instances, do not use this strategy for data synchronization jobs. Using this strategy when synchronizing data makes it possible to return a result with an earlier, incorrect version of the data.</p></li><li><p>Use this strategy to parallelize execution of batch job instances if you are certain that no record depends on the processing of another record within the Batch Job component, such as a case in which your batch job retrieves only new records from a database.</p></li></ul></div></li></ul></div></div></td></tr><tr><td><div><div><p>Job Instance ID</p></div></div></td><td><div><div><p><code>jobInstanceId</code></p></div></div></td><td><div><div><p>Optional expression that provides a recognizable, unique name for the batch instance. To make the name unique, you must use a DataWeave expression, for example, <code>#[jobInstanceId="#['Job From ' ++ now() as String {format: 'dd/MM/yy hh:mm:ss'}]</code> within this field. If an expression returns the same value more than once, Mule throws an error with a message such as, <code>Batch Job 'my-batch-job-example' already has an instance with id 'my-repeated-id'</code>. The Batch Job component uses this value to create a new identifier each time the Mule flow reaches the Batch Job component. If you do not provide an ID, Mule automatically creates a UUID that serves as the identifier for each batch job instance.</p></div></div></td></tr><tr><td><div><div><p>Batch Block Size</p></div></div></td><td><div><div><p><code>blockSize</code></p></div></div></td><td><div><div><p>Number of records to process per record block. The default is <code>100</code> records per block. If fewer records are available for processing than the configured size, Mule sends a block of a smaller size to the record processors. Such a case can occur, for example, if there are 1010 records and the block size is 100 because the modulo is 10, not 0. The block size setting provides a way to tune performance, which is affected by the size of records in each block and the <code>maxConcurrency</code> setting.</p></div></div></td></tr><tr><td><div><div><p>Max Concurrency</p></div></div></td><td><div><div><p><code>maxConcurrency</code></p></div></div></td><td><div><div><p>Property that sets the maximum level of parallelism to allow when processing record blocks within the component. The default is twice the number of cores available in the CPUs. The capacity of the system running the Mule instance also limits concurrency. Like the <code>blockSize</code> setting, this setting provides a way to tune performance.</p></div></div></td></tr><tr><td><div><div><p>Target</p></div></div></td><td><div><div><p><code>target</code></p></div></div></td><td><div><div><p>To use the payload received by a batch job instance in a component that is located <em>after</em> the Batch Job component, you can provide a unique value for the <code>target</code> and use <code>payload</code> in the downstream component to gain access that payload. Without a target value, the input payload to the Batch Job component is not available in downstream components. For more information about target variables, see <a href="https://docs.mulesoft.com/mule-runtime/4.6/target-variables">Enrich Data with Target Variables</a>. There is no default value.</p></div></div></td></tr></tbody></table>

### Batch Job History (<batch:history />)

A batch job history configuration enables you to address a `No space left on device` error by
adjusting the amount of time that historical data for batch job instances persist in a temporary
Mule directory. This error can occur if the frequency or number of records to process is too high
for the available disk space or for a CloudHub worker of a given size to handle.

By default, the history of batch job instances persists for 7 days. A monitoring process
automatically removes that history when the period expires.

![Batch Job History fields in Studio](/mulesoft/current/en-us/mule-runtime/4.6/_images/batch-job-fields-history-studio.png)

The following table describes each field and includes the XML attribute for the field.

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Name</th><th>XML</th><th>Description</th></tr></thead><tbody><tr><td><div><div><p>Max Age</p></div></div></td><td><div><div><p><code>maxAge</code></p></div></div></td><td><div><div><p>Maximum age of job instances before they expire, for example: <code>expiration maxAge="10"</code>. Default: <code>maxAge=7</code>.</p></div></div></td></tr><tr><td><div><div><p>Time Unit</p></div></div></td><td><div><div><p><code>ageUnit</code></p></div></div></td><td><div><div><p>The unit of time to apply to the <code>maxAge</code>. Valid values: <code>DAYS</code>, <code>HOURS</code>, <code>MILLISECONDS</code>, <code>MINUTES</code>, and <code>SECONDS</code>. Default: <code>ageUnit=DAYS</code>.</p></div></div></td></tr></tbody></table>

Note that these fields are attributes to the `<batch:expiration/>` element, which is embedded within
`<batch:history/>`:

```xml
<batch:job ...>
  <batch:history >
    <batch:expiration maxAge="10" ageUnit="MINUTES" />
  </batch:history>
  <batch:process-records>
    <batch:step />
  </batch:process-records>
<batch:job />
```

The XML example is edited to focus on settings in the batch history fields.

## Batch Step Component (<batch:step/>)

The Batch Step component runs during the [Process phase](https://docs.mulesoft.com/mule-runtime/4.6/batch-phases#phase_process) of a batch job
instance, not during the Load and Dispatch or On Complete phases.

A minimum of one Batch Step component is required within a Batch Job component. In addition to
changing the name of a Batch Step component, you can configure _filters_ that determine which
records the component accepts for processing within the component. Any processors that you add to
the component act on records that the Batch Job component accepts.

![Batch Step fields in Studio](/mulesoft/current/en-us/mule-runtime/4.6/_images/batch-step-fields-studio.png)

The following table describes each field and includes the XML attribute for the field.

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Field Name</th><th>XML</th><th>Description</th></tr></thead><tbody><tr><td><p>Name</p></td><td><p><code>name</code></p></td><td><div><div><p>Configurable name for a Batch Step component. The default is <code>Batch_Step</code> for the first Batch Step component within a Batch Job component. Subsequent steps append a unique number to the name, following the sequence, <code>Batch_Step1</code>, <code>Batch_Step2</code>, and so on.</p></div></div></td></tr><tr><td><p>Accept Expression</p></td><td><p><code>acceptExpression</code></p></td><td><div><div><p>Optional DataWeave expression for a filter that determines whether to process a record in the component. If <em>both</em> the <code>acceptExpression</code> <em>and</em> the <code>acceptPolicy</code> values for a record evaluate to <code>true</code>, then the component accepts the record for processing. If not, the component skips the record, and the record becomes available to any downstream Batch Step components. Examples:</p></div><div><ul><li><p><code>acceptExpression="#[payload.age &gt; 21]"</code> for a payload with an <code>age</code> field</p></li><li><p><code>acceptExpression="#[not isBlank(payload.name)]"</code> for a payload with a <code>name</code> field.</p></li></ul></div></div></td></tr><tr><td><p>Accept Policy</p></td><td><p><code>acceptPolicy</code></p></td><td><div><div><p>Accepts a record for processing only if the policy evaluates to <code>true</code> for that record. You can base acceptance on the success or failure of a record to process successfully in <em>preceding</em> (upstream) Batch Step components within the Batch Job component. You can configure the component to process only records that processed successfully, only records that failed to process successfully, or all records, regardless of whether they processed successfully. The default is <code>NO_FAILURES</code>. Note that the <code>acceptPolicy</code> evaluates before any <code>acceptExpression</code> on the component, and the <code>maxFailedRecords</code> setting on the <a href="#ref_batch_job_general_fields">Batch Job component</a> takes precedence over <code>acceptPolicy</code> settings.</p></div><div><ul><li><p><code>NO_FAILURES</code>: Default setting. Accepts a record for processing if the record never failed to process successfully within a preceding Batch Step component or if there is no upstream Batch Step component.</p></li><li><p><code>ONLY_FAILURES</code>: Accepts a record for processing if the record failed to process successfully within a preceding Batch Step component.</p></li><li><p><code>ALL</code>: Accepts a record for processing regardless of processing successfully or failing within a preceding Batch Step component.</p></li></ul></div></div></td></tr></tbody></table>

## Batch Aggregator Component (<batch:aggregator />)

The Batch Aggregator component is an optional component that acts on an array of records.
Aggregation takes place during the [Process phase](https://docs.mulesoft.com/mule-runtime/4.6/batch-phases#phase_process) of a batch job
instance.

There is no default configuration for the Batch Aggregator component. If you use this component, you
must indicate whether to stream records (`streaming`) or pull records from the queue into separate
arrays of a fixed size (`size`), but not both. The settings are mutually exclusive. Random access to
streamed records is not possible.

The Batch Aggregator component accepts one or more processors for the message `payload`. However, it
is common to use a scope within the aggregator, such as [For Each](https://docs.mulesoft.com/mule-runtime/4.6/for-each-scope-concept) scope, to
iterate over each record so that the child processors within the scope can act on each record
individually.

The Batch Aggregator component also provides a way preserve the MIME type of records
(`preserveMimeTypes`), which is needed when accessing part of a record that you are aggregating as a
fixed size (`size`). See
[Preserving the MIME types of the Aggregated Records](https://docs.mulesoft.com/mule-runtime/4.6/batch-filters-and-batch-aggregator#preserving-the-mime-types-of-the-aggregated-records).
To learn about record processing within this component, see batch aggregation information in
[Process Phase](https://docs.mulesoft.com/mule-runtime/4.6/batch-phases#batch_aggregator_processing).

Note that some connectors, such as
[Anypoint Connector for Salesforce (Salesforce Connector)](https://docs.mulesoft.com/salesforce-connector/latest/salesforce-connector-reference#upsert)
and
[Anypoint Connector for NetSuite (NetSuite Connector)](https://docs.mulesoft.com/netsuite-connector/latest/netsuite-reference#upsert),
provide operations that can handle record-level errors without causing the batch aggregation process
to fail.

When using Batch Aggregators, consider the following limitations:

- When using a Batch Aggregator component, do not attempt to process Guava data types, such as
  `ImmutableMap`. Instead, use a Java `Map` to avoid serialization issues.

- The Batch Aggregator component does not support job-instance-wide
  [transactions](https://docs.mulesoft.com/mule-runtime/4.6/transaction-management). Within a Batch Step component, you can define a
  transaction that processes each record in a separate transaction. However, any transaction started
  by a Batch Step component ends before the Batch Aggregator component starts processing. Components
  cannot share a transaction because a transaction cannot cross the boundary between the components.

![Batch Aggregator fields in Studio](/mulesoft/current/en-us/mule-runtime/4.6/_images/batch-aggregator-fields-studio.png)

The following table describes each field and includes the XML element for the field.

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Field Name</th><th>XML</th><th>Description</th></tr></thead><tbody><tr><td><div><div><p>Display Name</p></div></div></td><td><div><div><p><code>doc:name</code></p></div></div></td><td><div><div><p>Configurable name for the component. Example: <code>doc:name="My Aggregator"</code>. The default name is <code>Batch Aggregator</code>.</p></div></div></td></tr><tr><td><div><div><p>Aggregator Size</p></div></div></td><td><div><div><p><code>size</code></p></div></div></td><td><div><div><p>Size of each array of records to process. There is no default size. The processor within the component must accept an array as input. The component continues pulling records from the batch stepping queue into arrays of the configured size until none are left. If the queue contains fewer records than the configured size, the component pulls those records into a smaller array for processing. You must set the Size (<code>size</code>) or Streaming (<code>streaming</code>) property but not both.</p></div><div><p>When using this configuration, the component does not wait for the Batch Step component to finish processing all records in an instance or batch. Once the set number of records is available in the stepping queue, the Batch Aggregator component pulls those records into an array and processes them.</p></div></div></td></tr><tr><td><div><div><p>Streaming</p></div></div></td><td><div><div><p><code>streaming</code></p></div></div></td><td><div><div><p>When <code>streaming=true</code>, the component acts on all records in a batch job instance and returns the results in an array of a size equal to the number of records in the instance. The batch size setting through the Batch Job component does not limit the number of records to process or return in the array. For example, assume that the aggregator returns the numeric IDs of a small batch job instance that processes only <em>ten</em> records of the form <code>{"id": 1}</code>, <code>{"id": 2}</code>, and so on. In this case, the output array for the instance contains ten IDs, such as <code>[1, 6, 2, 7, 3, 8, 4, 9, 5, 10]</code>, even if the <code>blockSize</code> on the Batch Job component is set to a number smaller than the number of records in the instance, such as <code>5</code>.</p></div><div><p>Processors can gain sequential access to the array of streamed records. Random access to the streamed records is not allowed because the aggregator does not preserve the processed records.</p></div><div><p>There is no default. You must set the Size (<code>size</code>) or Streaming (<code>streaming</code>) property but not both.</p></div></div></td></tr><tr><td><div><div><p>Preserve Mime Types</p></div></div></td><td><div><div><p><code>preserveMimeTypes</code></p></div></div></td><td><div><div><p>Defaults to <code>false</code>. To preserve the MIME type of records processed within this component, set this attribute to <code>true</code>. Processors that cannot determine the MIME type of data return output, such as <code>null</code>, or unreadable references to the payload, such as <code>[[B@e596d5c, [B@1c4c3b32], …​]</code>. Setting <code>preserveMimeTypes=true</code> is useful when this component is set to process a fixed number (<code>size</code>) of records.</p></div></div></td></tr></tbody></table>

When deciding whether to stream aggregated records, consider the following:

- SaaS providers often have restrictions on accepting streaming input.

- Batch streaming is often useful in batch processing when writing to a file such as CSV, JSON, or
  XML.

- Batch streaming affects the performance of your application because the array of records is stored
  in RAM, which can slow the pace of transaction processing. Though performance slows, the trade-off
  of streaming data can warrant its use.

- Streaming prevents random access to records because a streaming commit is a one-read, forward-only
  iterator. However, you can perform random access operations when streaming if you use a component
  like [For Each](https://docs.mulesoft.com/mule-runtime/4.6/for-each-scope-concept). For Each provides a separate array of records, which is
  accessible through its `records` variable.