Einstein Subject Line Tester Model Card

Person or Organization

Einstein in Salesforce Marketing Cloud Engagement

Model Date and Version

• October 2021
• Minor changes can occur throughout the release
• Major changes can occur and are communicated via release notes

Model Type

• Prediction, Gradient boosted regression trees, Hyper-parameter grid search

General Information

• Einstein Performance Tester for Subject Lines (SLT) is the Salesforce solution to predict email engagement rate based on the subject line content that you provide.
• Analyze a single subject line’s performance or compare up to 10 subject lines at once.
• SLT helps customers understand the potential impact of language used in subject lines and improve the overall engagement of their subscribers.
• We use the global data pool due to the limited quantity of data a single customer is likely to have for subject line engagement predictions. We use the global data pool with the customer’s own local data to train the model. Because of this global model, even customers who have small data samples can take advantage of the performance tester. Customers must opt in to global models to use the subject line tester, in which case they'll contribute to and benefit from global modeling.
• Customer subject line contents vary greatly, so we have an optimal pooling strategy that makes a personalized tradeoff between a customer’s local data and global data for each customer. To provide the best predictions for the target customers, we put global data into the model and customize the model. We customize by searching for the hyper-parameters that are best for each customer. The two primary cases are customers with relatively large subject line samples and customers with relatively small subject line samples that still meet sample size requirements. In the first case, local data from the customer can already contain rich insights and patterns. The sample size of the local data can be large enough for modeling. The prediction performance is reasonably good. To improve the model, we retrieve insights from the global data and feed the additional insights to the model. Then, we take advantage of local data that represents the customer best. In this case, local data still dominates the performance of the mode. In the second case, we can retrieve limited insights from the customer's local data and the sample size is small. Therefore, the model trained with the local data yields high prediction error, so we use the global data for help. In this case, global data dominates the model performance. In all cases, customers must opt in to use global models in order to use the subject line tester.
• The features used to train the model include historical observations and natural language processing (NLP) features. The features can be divided into three levels of extractions; Sentence level, Word level, and Symbol or Character level. Sentence level includes word count, term frequency-inverse document frequency (TD-IDF). Word level includes a call to action, immediacy keyword checker. Symbol or Character level includes emoji, question mark, exclamation marker, and character count.
• See Global Model Data in Salesforce Help for more information.