Einstein Content Selection and Testing Model Card

Person or Organization

Salesforce Einstein for Marketing Cloud Engagement

Model Date and Version

• August 2020
• Minor changes can occur throughout the release
• Major changes can occur and are communicated via release notes

Model Type

Reinforcement learning, multi-armed bandit algorithms

General Information

The model measures engagement through the click-to-open rate (CTOR) in real time and selects image content when the email is opened. As more opens occur, Einstein favors top-performing content, but the remaining content is still used at lower rates. In the multi-armed bandit algorithm, the trade-off of optimizing for the highest-performing content versus continuing to experiment with other content is referred to as “exploiting vs exploring.”

Data from each send or journey is a standalone experiment. Assets with the highest CTOR in one experiment are reevaluated in subsequent experiments.

When using the model in a single batch send, stagger the send time to optimize the exploit phase. Einstein collects the most engagement data before new open events. To stagger sends automatically, use Einstein Send Time Optimization.

If a large portion of your audience uses Apple Mail Privacy Protection, Einstein needs a larger amount of email opens before providing the audience engagement data, resulting in a longer exploration phase. The longer phase could negatively impact your engagement rate compared to content sent without Einstein. To counter this effect, stagger your sends.

To personalize content in the Einstein Content Selection catalog, use the profile sync, content metadata, and Must Match attribute rules. These rules narrow which catalog content that an individual can be exposed to in the experiment. If the personalization rules remove content from the experiment, the remaining content that has the highest CTOR is the content to exploit.

Licenses

Einstein Content Selection and Einstein Content Testing are available to Marketing Cloud Engagement customers with these editions.

• Pro
• Corporate
• Enterprise
• Enterprise+

Primary Intended Uses

• Improve email engagement by experimenting and optimizing for the highest performing content.
• Personalize and manage the selected content using the provided Einstein Content Selection business rules.
• Use a simplified approach for how tested content is selected using Einstein Content Testing. ECT uses the same Einstein algorithm with traditional A/B/n testing, but simplified and without personalization or other business rules to manage how content is selected.

Out-of-Scope Use Cases

• Don’t use Einstein Content Selection or Testing for text content in emails or subject line testing because Einstein content runs open-time experiments.
• Don’t use Einstein Content Selection or Testing to test top-performing content across different sends or journeys.

Model Input

The Einstein model that powers Content Selection and Content Testing analyzes content performance for each send or journey.

Model Output

Content to display for a given email open event.

Groups

The model doesn’t include demographic data or other data purchased from third-party data providers.

Environment

The model is trained and deployed in the Marketing Cloud Engagement environment.

Model Performance Measures

Model performance metrics include data such as click-to-open rate (CTOR) and click-through rate (CTR). All metrics are aggregated and anonymized.