--- schema_version: "1" id: 6e923377935cbf174aa06b584d1df62360aedca889dca4f64e11f05a26774386 title: Protobuf Format collection: mulesoft version: current locale: en-us portal_url: https://docs.salesforce.com/mulesoft/current/en-us/dataweave/2.6/dataweave-formats-protobuf.md canonical_url: https://docs.mulesoft.com/dataweave/2.6/dataweave-formats-protobuf source_site: docs.mulesoft.com content_type: documentation content_hash: 85121ff380f2776f078c0847b96392fba0bd33283b9762592da813cc95f10741 description: MuleSoft Documentation Site release: "2.6" latest: false --- --- product: DataWeave version: 2.6 keywords: format, protobuf, application/protobuf, application/x-protobuf, application/protobuf --- # Protobuf Format > For the full documentation index, see: https://docs.mulesoft.com/llms.txt MIME type: `application/protobuf` ID: `protobuf` Protobuf is a data format that can be mapped to DataWeave values natively. From a user perspective, always specify the location of the descriptor file and the fully qualified name of the message for DataWeave to read or write. When specifying the configuration properties, use the `descriptorUrl` and the `messageType` properties, respectively. ## Example: Specify the Descriptor when Reading a Protobuf Message The following example shows how to specify the descriptor location and the message type to parse a Protobuf message and transform it into JSON. ### Schema The following schema specifies the protocol used in this example. ```protobuf syntax = "proto3"; package examples.descriptor; message MyMessage { int32 myInt = 3; bool myBool = 13; string myString = 23; } ``` ### Input The Protobuf message serves as input payload to the DataWeave source. It contains a `myInt` field with the value `42`, a `myBool` field with the value `false`, and a `myString` field with the value `DW + Proto`. We omit showing the Protobuf messages since their representation is not user-friendly. ### Source The DataWeave script reads a Protobuf message that has an `int` field, a `bool` field, and a `string` field, and transforms the input into JSON format. `messageType` points to the fully qualified name of the message being read, in this case `examples.descriptor.MyMessage`. `descriptorUrl` points to the descriptor, a compiled version of the `.proto` schema previously presented. ```dataweave %dw 2.0 output application/json input payload application/x-protobuf messageType='examples.descriptor.MyMessage',descriptorUrl="descriptors/examples.dsc" --- payload ``` ### Output The DataWeave script outputs the following JSON object containing the three values. ```json { "myInt": 42.0, "myBool": false, "myString": "DW <3 Proto" } ``` ## Protobuf Features Protobuf supports the following features which are not common in other formats. ### Enumerations [Protobuf enumerations](https://protobuf.dev/programming-guides/proto3/#enum), or enums, are read as a `String` with a particular schema that specifies the enum index. When parsing a `proto` message, the schema is used to extract the label of the enum value. If many labels share the same index, the first one is used. If the index does not have a matching label, the special label `"-UNRECOGNIZED"` is used. When writing a `proto` message, the schema specifies the protocol used to get the index corresponding to the given label. ### Example: Use DataWeave to Write Protobuf Enumerations The following example shows how to generate a `proto` message with some enum values, given a particular schema. #### Schema The following schema specifies the protocol used in this example. ```protobuf syntax = "proto3"; package examples.enumerations; message Langs { enum Languages { DataWeave = 0; Scala = 2; Java = 343049039; } Languages okayLanguage = 10; Languages bestLanguage = 11; } ``` #### Source The DataWeave script outputs a Protobuf message containing `okayLanguage` and `bestLanguage` values. ```dataweave %dw 2.0 output application/x-protobuf messageType='examples.enumerations.Langs',descriptorUrl="descriptors/examples.dsc" --- { okayLanguage : "Scala", bestLanguage : "DataWeave" } ``` #### Output The DataWeave script outputs a Protobuf message with two fields specified with the encoded values. ### Example: Use DataWeave to Read Protobuf Enumerations The following example shows how to read a `proto` message with some enum values, given a particular schema, and what to expect when the value present in the message is not specified in the schema. #### Schema The following schema specifies the protocol used in this example. ```protobuf syntax = "proto3"; package examples.enumerations; message Langs { enum Languages { DataWeave = 0; Scala = 2; Java = 343049039; } Languages okayLanguage = 10; Languages bestLanguage = 11; } ``` #### Input The Protobuf message which serves as input payload to the DataWeave source is not shown since it’s binary. It contains a `okayLanguage` field with enum index 3, an index not specified on the schema, while the `bestLanguage` field has the expected value. #### Source The DataWeave script just reads a Protobuf message and outputs it as a DataWeave output. ```dataweave %dw 2.0 input payload application/x-protobuf messageType='examples.enumerations.Langs',descriptorUrl="descriptors/examples.dsc" output application/dw --- payload ``` #### Output The DataWeave script shows how the Protobuf message input is represented in the DataWeave (`dw`) format. ```dataweave { okayLanguage: "-UNRECOGNIZED" as String {protobufEnumIndex: 3}, bestLanguage: "DataWeave" as String {protobufEnumIndex: 0}, } ``` ### Oneof [Oneof](https://protobuf.dev/programming-guides/proto3/#oneof) fields, a Protobuf particularity, are represented as regular fields on DataWeave. When writing a `proto` message, a particular schema specifies what the DataWeave script needs to validate. If more than one of the fields defined is present, the script fails. ### Example: An Invalid Attempt to Write Two Exclusive Fields The following example shows what happens when you try to write two exclusive fields according to the schema. #### Schema The following schema specifies the protocol used in this example. ```protobuf syntax = "proto3"; package examples.oneof; message ThisOrThat { oneof thisOrThat { bool this = 2; bool that = 4; } } ``` #### Source The DataWeave script outputs a Protobuf message containing both `this` and `that` fields set. ```dataweave %dw 2.0 output application/x-protobuf messageType='examples.oneof.ThisOrThat',descriptorUrl="descriptors/examples.dsc" --- { this: true, that: false, } ``` #### Output The DataWeave script outputs a `ProtoBufWritingException` specifying which `oneof` is being wrongly used. ### Repeated Fields Since DataWeave admits repeated fields, Protobuf repeated fields are matched to DataWeave repeated fields, and vice versa. Note that the DataWeave object being written has to match the schema being used. If the schema does not specify a field as repeated and the DataWeave script has that field more than once, the script fails. ### Example: Transform a JSON Array to a Protobuf Repeated Field The following example shows how to generate a `proto` message with a repeated field obtained from a JSON array. #### Schema The following schema specifies the protocol used in this example. ```protobuf syntax = "proto3"; package examples.repeated; message People { repeated string names = 1; } ``` #### Input The JSON input serves as the payload to the DataWeave source. ```json { "names": ["Mariano", "Shoki", "Tomo", "Ana"] } ``` #### Source The DataWeave script uses the `reduce` function to generate a new field for each name in the array. ```dataweave %dw 2.0 output application/x-protobuf messageType='examples.repeated.People',descriptorUrl="descriptors/examples.dsc" --- reduce(payload.names, (item, acc = {}) -> acc ++ {names: item}) ``` #### Output The DataWeave script outputs a Protobuf message with the repeated field containing all the names from the JSON array. ### Example: Transform a Protobuf Repeated Field to a JSON Array The following example shows how to transform a `proto` message with a repeated field to a JSON array. #### Schema The following schema specifies the protocol used in this example. ```protobuf syntax = "proto3"; package examples.repeated; message People { repeated string names = 1; } ``` #### Source The DataWeave script outputs a JSON message containing an array of names. ```dataweave %dw 2.0 input payload application/x-protobuf messageType='examples.repeated.People',descriptorUrl="descriptors/examples.dsc" output application/json --- names: payload.people.*names ``` #### Output The DataWeave script outputs a JSON message containing an array of names. ```json { "names": ["Mariano", "Shoki", "Tomo", "Ana"] } ``` ### Unknowns Protobuf offers capabilities for forward and backward compatibility of protocols. In order to achieve this, readers and writers accept [unknown fields](https://protobuf.dev/programming-guides/proto3/#unknowns) on messages. DataWeave adapts to this functionality by using certain key names. When reading a Protobuf message, if a field not present in the schema is found, it is read into something similar to `"-35": 111111 as Number {wireType: "Varint"},`, where `"-35"` means that the field index is `35`, and `wireType: "Varint"` specifies the wire type the field has in the message. The `wireType` can be `"Varint"`, `"64Bit"`, `"LengthDelimited"`, `"Group"`, or `"32Bit"`. ### Semantic Parsing (Or Commonly Used Message Types) Protobuf offers a collection of [commonly used message types](https://protobuf.dev/reference/protobuf/google.protobuf/). DataWeave parses some of these as the value they represent instead of as the underlying message. For example, a `google.protobuf.Duration` is read into DataWeave as a `Period`, while a `google.protobuf.NullValue` is read as `Null`. The following table describes the correspondence between Protobuf types and DataWeave types.
Protobuf typeDataWeave type

BoolValue

Boolean

BytesValue

Binary

DoubleValue

Number

Duration

a Duration Period

Empty

{}

FloatValue

Number

Int32Value

Number

Int64Value

Number

ListValue

Array

NullValue

Null

StringValue

String

Struct

Object

Timestamp

LocalDateTime

UInt32Value

Number

UInt64Value

Number

Value

ProtoBufValue

Where: ```dataweave type ProtoBufValue = Null | Number | String | Boolean | { _?: ProtoBufValue } | Array ``` ### Maps [Protobuf maps](https://protobuf.dev/programming-guides/proto3/#maps) enable you to have a structure without a predefined set of keys, but with every field sharing the same value type. A `map` is mapped to a DataWeave object with the keys represented as strings and the values mapped to their corresponding value. When writing a `proto` message, the key is casted to the `keyType` specified on the descriptor. If it’s not possible to execute the cast, the script fails. ## Compiling Schemas into Descriptors DataWeave is not able to directly use `*.proto` files and expects an already compiled version called descriptor. Generate descriptors by using the `protoc` compiler as in `protoc --descriptor_set_out=./out.dsc file1.proto file2.proto …​`, where `out.dsc` is the output file for the descriptor (the one that DataWeave expects on the `descriptorUrl` property), while `file1.proto` and `file2.proto` are the actual protocol specifications that the descriptor needs to compile. ## Configuration Properties DataWeave supports the following configuration properties for this format. ### Reader Properties This format accepts properties that provide instructions for reading input data.
ParameterTypeDefaultDescription

bufferSize

Number

8192

Size of the buffer writer, in bytes. The value must be greater than 8.

deferred

Boolean

false

Generates the output as a data stream when set to true, and defers the script’s execution until the generated content is consumed.

Valid values are true or false.

descriptorUrl (Required)

String

''

The URL for the ProtoBuf descriptor. Valid values are classpath://, file://, or http://.

messageType (Required)

String

null

The message type’s full name taken from the given descriptor, including the package where it’s located.

### Writer Properties This format accepts properties that provide instructions for writing output data.
ParameterTypeDefaultDescription

bufferSize

Number

8192

Size of the buffer writer, in bytes. The value must be greater than 8.

deferred

Boolean

false

Generates the output as a data stream when set to true, and defers the script’s execution until the generated content is consumed.

Valid values are true or false.

descriptorUrl (Required)

String

''

The URL for the ProtoBuf descriptor. Valid values are classpath://, file://, or http://.

messageType (Required)

String

null

The message type’s full name taken from the given descriptor, including the package where it’s located.

## Supported MIME Types This format supports the following MIME types.
MIME Type

application/protobuf

application/x-protobuf