---
schema_version: "1"
id: a2e8fd46472a5770bd024cc85c187e7cae7eaf88876eaba7c6868a743fd26a72
title: Write and Validate Custom Scripts
collection: developer
version: current
locale: en-us
portal_url: https://docs.salesforce.com/developer/current/en-us/data-cloud/data-cloud-code-ext/guides/author-custom-script.md
canonical_url: https://developer.salesforce.com/docs/data/data-cloud-code-ext/guide/author-custom-script.html
source_site: developer.salesforce.com
content_type: documentation
content_hash: 0c1c089ca9c3166bbb1d81471f92fb3eb908e2897151df087777a52d059c456f
products:
  - Data 360 Code Extension Guide
  - Data 360 Developer Documentation
---
# Write and Validate Custom Scripts

Write custom scripts for batch data transforms in a standard Python environment by using your preferred IDE and validate them locally against a sandbox.

| Edition Table                                                                                                                                                                                                 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Available in: **Developer**, **Enterprise**, **Performance**, and **Unlimited** Editions. See [Data 360 edition availability](https://help.salesforce.com/s/articleView?id=sf.c360_a_dc_editions.htm&type=5). |

| Permission Sets Needed                                               |                                                        |
| -------------------------------------------------------------------- | ------------------------------------------------------ |
| To write custom scripts and validate them locally against a sandbox: | Permission set: <ul><li>Data Cloud Architect</li></ul> |

## Prerequisites

1. Set up your Data 360 environment.
   1. Verify that you have access to a Data 360 sandbox with appropriate permission sets. If you don’t have a sandbox, create one. See [Create a Data 360 Sandbox](https://help.salesforce.com/s/articleView?id=data.c360_a_data_cloud_sandbox_create.htm&type=5).

   2. Enable Code Extension in Data Cloud Setup using the Feature Manager. See [Enable Data 360 Features in Feature Manager](https://help.salesforce.com/s/articleView?id=data.c360_a_feature_manager.htm&language=en_US&type=5).

   3. Verify that you ingested data into your sandbox to create the DLOs or DMOs that your script references.

2. Set up Salesforce CLI for code extension and create a script package. For more information, see [Set Up Salesforce CLI for Code Extension](https://developer.salesforce.com/docs/data/data-cloud-code-ext/guide/set-up-sdk.md).

## Best Practices

- Work exclusively with either DLOs or DMOs in a single script.
- Add any additional Python dependencies to `requirements.txt`.
- Use only pip-installable dependencies. Libraries that require OS-level installations or system configurations don’t work in Data 360.
- Use clear function names and comments, and handle errors gracefully.
- Test your logic frequently in your selected environment.
- Use logging for debugging and monitoring.

## Column and Object Name Casing in Custom Scripts

Column names and object API names use different casing in your script.

### Column Names

Reference column names in lowercase, even though each field keeps its original casing in the Data 360 UI and in the field's API name. For example, use `unitprice__c` for the `UnitPrice__c` field. A mixed-case column name returns a `column does not exist` error.

Use lowercase column names in every PySpark expression that references a column, such as `col()`, `withColumn()`, `select()`, and `drop()`.

When you test locally, the script returns the same column names that Data 360 returns. Columns that resolve during local testing also resolve after you deploy the script.

**Incorrect:**

```python
df = df.withColumn("UnitPrice__c", upper(col("UnitPrice__c")))
```

**Correct:**

```python
df = df.withColumn("unitprice__c", upper(col("unitprice__c")))
df = df.drop("kq_id__c")
```

### Object API Names

Pass DLO and DMO API names exactly as they appear in Data 360, such as `Account_Home__dll`.

**Example:**

```python
df = client.read_dlo("Account_Home__dll")
```

**important:** In code extension, you can't use a DLO or DMO that one batch data transform writes to as an input object for another batch data transform.

## Data Writing Modes

Use one of these write modes when writing to DLOs or DMOs in your script.

| Write Mode              | Description                                                                                                            | Example                                                                                |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| **APPEND**              | Adds new records to the target object without affecting existing data.                                                 | `client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.APPEND)`              |
| **OVERWRITE**           | Replaces all existing data in the target object with new data.                                                         | `client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.OVERWRITE)`           |
| **MERGE**               | Updates existing records and adds new records based on key fields.                                                     | `client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.MERGE)`               |
| **MERGE_UPSERT_DELETE** | Updates, adds, or deletes each record based on the `_merge_record_type` column that you add to your DataFrame.         | `client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.MERGE_UPSERT_DELETE)` |

### Mark Records to Upsert or Delete

Add the `_merge_record_type` column to your DataFrame and set it to `UPSERT` for records to update or add, or `DELETE` for records to remove. `withColumn()` requires a column expression rather than a plain string, set the value with `lit()`.

A single write operation can contain both upserts and deletes. Data 360 matches each record to the target object by primary key. A record set to `UPSERT` updates the matching record, or adds the record if no match exists. A record set to `DELETE` removes the matching record; If no match exists, the delete operation does nothing.

**important:** Your DataFrame must meet these requirements:

- Include the `_merge_record_type` column, or the write fails with an `InvalidDatasetForMergeUpsertDelete` error.
- Set `_merge_record_type` to `UPSERT` or `DELETE` on every record. Data 360 skips records that are set to any other value.

**Example**

```python
from datacustomcode.client import Client
from datacustomcode.io.writer.base import (
    MERGE_RECORD_TYPE_COLUMN,
    MergeRecordType,
    WriteMode,
)
from pyspark.sql.functions import col, lit

client = Client()
source_df = client.read_dlo("Source_DLO__dll")

# Select the primary key and at least one other field in both DataFrames.
upserts = (
    source_df.filter(col("status__c") != "obsolete")
    .select(col("id__c"), col("name__c"))
    .withColumn(MERGE_RECORD_TYPE_COLUMN, lit(MergeRecordType.UPSERT))
)
deletes = (
    source_df.filter(col("status__c") == "obsolete")
    .select(col("id__c"), col("name__c"))
    .withColumn(MERGE_RECORD_TYPE_COLUMN, lit(MergeRecordType.DELETE))
)

client.write_to_dlo(
    "Target_DLO__dll",
    upserts.unionByName(deletes),
    write_mode=WriteMode.MERGE_UPSERT_DELETE,
)
```

Use the same pattern with `client.read_dmo()` and `client.write_to_dmo()` when your script uses DMOs.

### Authoring in Standard Python Virtual Environment

1. Go to your code extension package folder.

   ```bash
   cd </path/to/your/package>
   ```

2. Create and activate a Python virtual environment in the package folder.

   **macOS and Linux**

   ```bash
   python3.11 -m venv venv
   source venv/bin/activate
   ```

   **Windows (PowerShell)**

   ```bash
   py -3.11 -m venv venv
   .\venv\Scripts\Activate.ps1
   ```

   **Windows (Command Prompt)**

   ```bat
   py -3.11 -m venv venv
   venv\Scripts\activate.bat
   ```

3. Add required dependencies to `requirements.txt`.

   ```bash
   # examples
   pandas>=1.5.0
   numpy>=1.21.0
   ```

4. Install the dependencies.

   **macOS and Linux**

   ```bash
   pip install -r requirements.txt
   ```

   **Windows**

   ```
   python -m pip install -r requirements.txt
   ```

**note:** To add new dependencies during development, always add them to `requirements.txt` and then install by using this command. This process ensures that all dependencies are included in your deployment package.

5. Write your script in `payload/entrypoint.py` to reference the ingested data.

6. Test your script.

   ```bash
   sf data-code-extension script run --entrypoint <path_to_entrypoint> --target-org <org_alias>
   ```

   - `--entrypoint <path_to_entrypoint>` (required): Path to your script entry point (typically `./payload/entrypoint.py` in the default package layout).
   - `--target-org <org_alias>` (required): The alias against which you’re testing your script.

   **Example:**

   ```bash
   sf data-code-extension script run --entrypoint ./payload/entrypoint.py --target-org myorg
   ```

7. Optionally, control the number of rows returned during local testing.

   By default, Data 360 queries return 1,000 rows when you test locally. To validate against a larger dataset, raise the row limit. To iterate faster on a smaller sample, lower it. Set the row limit in `entrypoint.py` before your script reads any data.

   1. Import `config` in your `entrypoint.py`.

      ```python
      from datacustomcode.config import config
      ```

   2. Set the row limit to the number of rows you need. For example, to return 5,000 rows:

      ```python
      config.reader_config.options["default_row_limit"] = 5000
      ```

      To remove the limit entirely and fetch all records, set the value to `None`:

      ```python
      config.reader_config.options["default_row_limit"] = None
      ```

   **important:** If you package and upload your entrypoint with this setting, the package includes the row limit, which then applies when run in Data 360. Remove or reset the row limit before packaging for deployment.

8. Update the Data 360 configuration file after changing your script.

   ```bash
   sf data-code-extension script scan --entrypoint ./payload/entrypoint.py
   ```

   **important:** The **`sf data-code-extension script scan`** command doesn't work as intended for DMO-to-DMO transforms. You must manually craft the `config.json` file. See [Configure `config.json` for DMO-to-DMO Transforms](https://developer.salesforce.com/docs/data/data-cloud-code-ext/guide/configure-dmo-schema.md).

## Next Step

[Deploy a Custom Script to Data 360 Sandbox by Using UI](https://developer.salesforce.com/docs/data/data-cloud-code-ext/guide/deploy-custom-code.md).

## See Also

- [Considerations When Writing Code Extension](https://developer.salesforce.com/docs/data/data-cloud-code-ext/guide/considerations-when-writing-custom-code.md)