append Transformation

The append transformation combines rows from multiple datasets into a single dataset.

Note To append rows in a Data Prep recipe, see Append Node: Stack Rows from Different Sets of Data. To do it in a Data Prep Classic recipe, see Add Rows in a Recipe with Append.

Consider the following rules when using this transformation.

This transformation does not remove duplicate records.
All input dataset must have the same structure—the corresponding columns must be in the same order and have the same name and field type.

Example

Let’s look at an example. Each month, you create a dataset that contains the month’s sales targets. Now, you want a holistic view of sales targets for all months. To do that, you create the following dataflow to merge the existing dataset into a single dataset.

Dataflow	Append Node in Dataflow

Here’s the JSON dataflow.

{
  "Extract SalesTargets_Month3": {
    "action": "edgemart",
    "parameters": {
      "alias": "SalesTargets_Month3"
    }
  },
  "Extract SalesTargets_Month2": {
    "action": "edgemart",
    "parameters": {
      "alias": "SalesTargets_Month2"
    }
  },
  "Extract SalesTargets_Month1": {
    "action": "edgemart",
    "parameters": {
      "alias": "SalesTargets_Month1"
    }
  },
  "Append SalesTargets_Quarter1": {
    "action": "append",
    "parameters": {
      "enableDisjointedSchemaMerge": false,
      "sources": [
        "Extract SalesTargets_Month1",
        "Extract SalesTargets_Month2",
        "Extract SalesTargets_Month3"
      ]
    }
  },
  "Register AllSalesTargets": {
    "action": "sfdcRegister",
    "parameters": {
      "name": "All Sales Targets",
      "alias": "AllSalesTargets",
      "source": "Append SalesTargets_Quarter1"
    }
  }
}

After you create the single dataset, you can use date filters to analyze the sales targets by month, quarter, or year.

Note The append transformation doesn’t remove duplicate rows.

Append Datasets with Different Schema

By default, the input dataset must have the same structure—the corresponding columns must have the same name and field type. For example, let's say you want to append your Canada opportunities to your U.S. opportunities.

US Sales	Canada Sales

Here, the column names are different, and the Canada data has a Currency column. For the dataflow not to fail, select Allow disjoint schema in the append node in the dataflow editor. If you're working in the dataflow JSON, add the enableDisjointedSchemaMerge parameter and set its value to true.

Append Node in Dataflow Editor	Append Node in Dataflow JSON
	`"Append US & CA Sales": { "action": "append", "parameters": { "enableDisjointedSchemaMerge": true, "sources": [ "Extract US Sales", "Extract CA Sales" ] } },`

When you run the dataflow, the data is merged without the dataflow failing.

Table of combined US and Canada opportunities

The append transformation adds all columns to the dataset and merges values in columns with the same name, such as Name. It also adds null values in a column for rows that didn’t previously have that column, such as in the Currency column.

Note If null measure handling in datasets isn’t enabled for your org, append adds zeros in a column for rows that didn’t previously have that column.

append Parameters
When you define an append transformation in the dataflow JSON, you set the action attribute to append and specify the parameters.