Einstein Messaging Insights Model Card

Person or Organization

Salesforce Einstein in Marketing Cloud Engagement

Model Date and Version

• May 2020
• Minor changes can occur throughout the release
• Major changes can occur and are communicated via release notes

Model Type

Regression, Causation analysis, Time-series analysis

General Information

Einstein Messaging Insights is an anomaly detection system that surfaces noteworthy facts about ongoing marketing email campaigns running in Marketing Cloud Engagement. The model can detect and compare campaigns with similar attributes, performance changes over time, and relationship changes between activities for complex orchestration. The model comprises multiple decision tree regression models, time series-based predictions, and graph analytics. Einstein trains each customer’s model only using that customer’s data.

Constraints

• A minimum of 10 email batches is required to model for batch insights.
• A minimum of 7 days of history is required to model for journey, journey activity, and frequent automation insights.

License

Einstein Messaging Insights is available to Salesforce Marketing Cloud Engagement customers with any of these editions.

• Corporate Edition
• Enterprise Edition
• Enterprise+ Edition

Primary Intended Uses

The Einstein Messaging Insights model powers Einstein notifications and the Einstein Messaging Insights dashboard hub. These tools are designed for marketing professionals to see anomalously high or low engagement rates of email sends compared to expected values. The feature notifies marketers of anomalies so that they can further investigate their journeys and email sends.

Out-of-Scope Use Cases

Anything other than the primary use case is out of scope and not recommended. For example, when an anomaly is raised, inferring that related metrics like click-to-open rate (CTOR) are also anomalous isn’t an intended use case.

Model Input

Einstein Messaging Insights analyzes up to 90 days of historical engagement patterns of the Enterprise subscribers. The engagement history includes these factors.

• Email sends, bounces, and engagement events including open, click, and unsubscribe, spam complaints, and associated timestamps
• Data and metadata about customer sending patterns, including how campaigns are executed. This data includes batch sends, Journey Builder sends, and Automation Studio sends, transactional sends, and triggered sends
• Data and metadata about customer sending patterns, including past and future automations running in Automation Studio
• Pre-personalization subject lines are analyzed with unresolved AMPscript
• Email domain

The engagement history that Einstein Messaging Insights analyzes excludes these factors.

• Data purchased or collected from third parties
• Demographic Data, which is typically stored in SFMC as data extensions or subscriber or contact attributes
• Specific content within the email template or rendered email body

Model Output

The Einstein Messaging Insights model outputs these insights for three types of engagement (clicks, opens, and unsubscribes) daily. Each insight is generated based on observed engagement rate and incorporates upper and lower confidence bounds. All insights include an anomaly degree, which measures the severity of the insight’s impact.

• Batch insights, which consider anomalous batches sent over the past 7 days.
• Batch insights include top contributing signals with corresponding signal strengths.
• Journey insights, which consider anomalous journeys whose engagement rate is outside of the expected range.
• Journey activity insights, which consider anomalous journeys whose activities’ engagement rate’s overall change is outside of the expected range.
• Automation insights, which consider anomalous frequent automations whose engagement rate is outside of the expected range.

Groups

The model doesn’t include any demographic data or other data purchased from third-party data providers. The model uses engagement data such as opens, sends, and patterns.

Environment

The model is trained and deployed in the Salesforce Marketing Cloud Engagement environment.

Model Performance Measures

We evaluate and monitor model performance metrics to ensure and improve the quality of the model. Model performance metrics include metrics like model fitness score (root mean squared error) and confidence-bound tightness.