Ensure that you're getting the most out of your batch data transforms by keeping the following guidelines and recommendations in mind.
- Best Practices for Batch Data Transforms
Keep your Batch Data Transforms fast and reliable by filtering early, keeping data narrow, and avoiding common design pitfalls like exploding joins and data skew. These guidelines cover planning and testing, node selection, join key design, and recommended limits — with the biggest performance gains coming from how you structure your data flow before it reaches joins and aggregations. - Keep DMO Mappings When Deploying Data Transforms via Data Kits
When you deploy a Data Transform using a Data Kit, the mappings between Output Data Lake Objects (DLOs) and Data Model Objects (DMOs) aren't automatically included. To carry over these mappings, explicitly add the Output DLOs to the Data Kit. - Data Space Filters in Data Transforms
Data space filter behavior in data transforms differs based on whether the transform uses DLO or DMO inputs. This distinction helps you anticipate which records appear in preview and which records the transform processes when it runs. - Preview Results in a Batch Data Transform
Catch mistakes early by reviewing the results of each node and transformation that you add to a batch data transform. Each user can initiate up to 4,000 previews per hour. The preview shows a sample of the latest available data, up to 5,000 rows. - Batch Data Transforms Limitations
Batch data transforms have the following limitations. - Edit a Batch Data Transform
Make changes to your batch data transforms and save the results. - Delete a Batch Data Transform
Delete any batch data transforms that you no longer need. Deleting a batch data transform doesn’t delete its target data.