A/B Test Segments for B2C Commerce

Distribute test segments across a maximum of 12 simultaneous A/B tests, as long as the total segments for all active tests don’t exceed 12. Each A/B test can have up to five unique test segments, one of which is always the control group. Each test segment targets a portion of the total test participants. The control group receives the same storefront experience as if the A/B test didn’t exist.

Other segments receive a modified storefront experience with one or more changes to the control group experience. These changes are only visible to a particular segment. They can be from one or more built-in Salesforce B2C Commerce A/B testing experiences, and script customizations.

Promotions test which bargains generate the best response.
Content slots test the effectiveness of graphics and content.
Sorting rules look at search results to see their impact.

A customer can only participate in one A/B test at a time.

We recommend that you test one experience with a new slot content and promotion against one with a different slot content and experience.

Note

When you run A/B tests, avoid using existing promotions, slot content, and sorting rules because they’re included in the controls. Instead, create an experience to test against the controls.

Test Group Effects on the Storefront Experience

Each A/B test group can have effects that automatically change the experience for its participants:

Within the test...	Effect...
Include a promotion.	The test makes a promotion available that is otherwise unavailable. When you include a promotion in a test, specify a rank. The rank you assign doesn’t change the rules governing the exclusivity and qualifying products of a promotion. These rules still apply, and can cause another promotion to take precedence over one included for an A/B test. You can assign each promotion to only one A/B test segment one time. Assigned promotions run for the length of the test.
Include a slot configuration.	The test makes a slot configuration available that is otherwise unavailable. When you include a slot configuration in a test, the system provides a rank. Assign each slot configuration to only one A/B test group at a time. Slot configurations run for the length of the test.
Override a sorting rule.	Overrides the default sorting rule for a category that normally applies. When you assign a category sorting rule to an A/B test, it replaces what you selected on the Sorting Rule tab on the Category page. Where B2C Commerce ignores a category sorting rule, it ignores the rule whether you selected it via the Sorting Rule tab or an A/B test. For example, if you set a sorting rule with the ProductSearchModel API or the Search pipelet. B2C Commerce uses that explicit rule and ignores the default rule, regardless of whether you set the default on the Category page or applied it as an effect for a test segment.

Within the test...

Effect...

Include a promotion.

The test makes a promotion available that is otherwise unavailable. When you include a promotion in a test, specify a rank. The rank you assign doesn’t change the rules governing the exclusivity and qualifying products of a promotion. These rules still apply, and can cause another promotion to take precedence over one included for an A/B test.

You can assign each promotion to only one A/B test segment one time. Assigned promotions run for the length of the test.

Include a slot configuration.

The test makes a slot configuration available that is otherwise unavailable. When you include a slot configuration in a test, the system provides a rank. Assign each slot configuration to only one A/B test group at a time. Slot configurations run for the length of the test.

Override a sorting rule.

Overrides the default sorting rule for a category that normally applies.

When you assign a category sorting rule to an A/B test, it replaces what you selected on the Sorting Rule tab on the Category page. Where B2C Commerce ignores a category sorting rule, it ignores the rule whether you selected it via the Sorting Rule tab or an A/B test. For example, if you set a sorting rule with the ProductSearchModel API or the Search pipelet. B2C Commerce uses that explicit rule and ignores the default rule, regardless of whether you set the default on the Category page or applied it as an effect for a test segment.