---
schema_version: "1"
id: 0fccc9ddfad644ad0533f8f949ab12feff6701cffa247020df2379827bbd5de0
title: Indexed Readers in DataWeave
collection: mulesoft
version: current
locale: en-us
portal_url: https://docs.salesforce.com/mulesoft/current/en-us/dataweave/2.4/dataweave-indexed-readers.md
canonical_url: https://docs.mulesoft.com/dataweave/2.4/dataweave-indexed-readers
source_site: docs.mulesoft.com
content_type: documentation
content_hash: ac2bf12c22a9cfbced462fdbf9592889f9c25afcda5f6bb09857f8313bfe30a1
description: MuleSoft Documentation Site
release: "2.4"
latest: false
---
---
product: DataWeave
version: 2.4
---

# Indexed Readers in DataWeave

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

In DataWeave, some readers of input data (such as XML, JSON, and CSV) support an indexed strategy to
avoid loading the whole document in memory while still allowing random access just like in the
in-memory strategy. DataWeave uses indexed readers when the input is larger than a certain
configurable threshold or if the `indexedReader` setting is set to `true`.

The trade-off for managing files that wouldn’t fit in memory is to write the files to a temporary
file in disk and index the document content beforehand, which also takes time and disk space.

Indexed readers can process input files up to 20 GB. For bigger files use the streaming strategy,
which is faster. Despite this mode being more limited in functionality, there’s no maximum input
size limitation for streaming readers.

The actual max input file size supported is difficult to estimate because it depends on the content
of the input.

The following list shows the limits on different parts of the input file:

- Max nesting depth: `4096`

- Max individual value size: `4 GB`

- Tokens: `​​2^32-1` (approximately 4 billion)  
  A token is either a key/value pair, object-start or array-start, that limits the max input file
  size. The amount of the max input file size is irrespective of the size of each token in the file.
  The bigger your average value length, the bigger your input file size can be because what is
  important is the amount of tokens, not their size.

The following examples show input file size samples:

In this example, the max input file size is between 40 GB and 50 GB:

```json
[
  {
    "name": "Mariano",
    "lastName": "Achaval"
  },
  ...
]
```

The following example is similar to the previous one but minified where the max file size is
approximately 30 GB:

```json
[{"name":"Mariano","lastName":"Achaval"},...]
```

The following example shows a minified array of numbers `1`. DataWeave supports only approximately 8
GB of such file:

```json
[1,1,1,1,1,1,...]
```

## Large Strings Management

When processing a `String` with a size larger than 1.5 MB, DataWeave automatically splits the value
in chunks to avoid out-of-memory issues. This feature works only with `JSON` and `XML` input data.

You can configure the threshold that DataWeave uses to determine when to process a string using this
splitting strategy by modifying the following system property:

```none
com.mulesoft.dw.max_memory_allocation
```

Note that using this feature has a negative impact on performance due to splitting strings and
accessing them through local storage. If your environment has enough resources to load all content
in memory, you can also disable this string management feature completely by setting the value of
the following system property to `false`:

```none
com.mulesoft.dw.buffered_char_sequence.enabled
```

## See Also

- [CSV Format](https://docs.mulesoft.com/dataweave/2.4/dataweave-formats-csv)

- [JSON Format](https://docs.mulesoft.com/dataweave/2.4/dataweave-formats-json)

- [XML Format](https://docs.mulesoft.com/dataweave/2.4/dataweave-formats-xml)

- [Supported DataWeave Formats](https://docs.mulesoft.com/dataweave/2.4/dataweave-formats)