A/B test campaign messages

What is it?

A/B testing in Campaign Composer allows you to test different variations of a campaign message—such as copy, links, media, or send times—to evaluate how subscribers engage with each version.

Campaign Composer supports A/B testing across both text (SMS) and email channels. Depending on your configuration, you can test a partial audience before sending the top-performing variation to the remaining audience, or test across 100% of your recipients. Once a test completes, Campaign Composer displays performance metrics, winner badges, statistical significance indicators, and side-by-side variant comparisons.

How do I use it?

Step 1: Set up a content or send time A/B test

1. Go to Campaigns and click + Create campaign.
2. Select Create an A/B test.
3. Choose the message channel 
4. Select what you want to test:
   • Content: Variations in text, media, or links.
   • Send time: Variations in delivery schedules.
5. Click Next.

Step 2: Configure campaign settings and target audience

1. Enter an internal Campaign name and Message name.
2. (Optional) Add up to five Tags to organize your campaign.
3. Click Edit under Audience to select or exclude subscriber segments.
4. Select delivery scheduling options under When to send:

Step 3: Configure test details and variations

Enter campaign copy and media for your initial variations. By default, two variations are created. Click + Create another variation to add more (up to 30 variations total).

Test details 

1. Under Test details, select your Audience Split:
   • Split traffic evenly: Choose a percentage of your total audience to receive the test variations (default is 20%). Select a Test duration (default is 2 hours) and a Winning metric (such as Click-Through Rate or Conversion Rate). After the test duration finishes, the winning variation automatically sends to the remaining audience.
     • Full audience test (100% split): Adjust the slider to 100% to send test variations across your entire audience without sending a subsequent winning message.

• Customize traffic split: Select an even audience split or enter custom distribution percentages among variations (must equal 100% of the test audience).

Send time A/B tests

1. Under Test details & send time, click Edit.
2. Define delivery schedules for Send time 1 and Send time 2. Click + Add variation to include additional send times.
3. If your message contains links, select a Test parameter (e.g., utm_campaign) to attach dynamic tracking tags to each variation for third-party analytics.

Step 4: Review and schedule

1. Click Send test message to preview how each variant appears on a mobile device or email client.
2. Click Review & schedule campaign.
3. Scheduled campaigns can be monitored on the main Campaigns page.

Step 5: View test results and statistical significance

Once a test finishes, you can review its performance from the Campaigns tab. The Campaigns list view Displays a Winner label next to the top-performing variant once the test completes. Note: Winner and Stat Sig labels will appear consistently for all campaigns sent after 8/11/26. 

Additional performance metrics are available for each A/B test as you drill in:

1. View all variants side-by-side: Click the top-level campaign name in the list to open the Campaign performance page. Here, you will find the side-by-side performance table to compare all variants across all captured metrics (Sends, Click-Through Rate, Conversion Rate, Revenue, Unsubscribes) in a single consolidated view.
   • Campaign A/B test performance page: Displays the Winner label. For non-"pick a winner" tests (test variations go to 100% of audience), the detailed metrics page also displays a Statistically Significant indicator next to the test metric when appropriate, confirming whether the winning result met statistical significance thresholds.

2. View each variant in detail: Click directly on an individual variant to open its dedicated, in-depth performance metrics breakdown.

Why should I use it?

Running structured A/B tests provides data to inform your messaging strategy:

• Identify effective content: Determine which images, copy, and urgency language generate higher engagement and conversion rates.
• Determine optimal send times: Test different times of day to establish when subscribers are most responsive.
• Review full performance side-by-side: View all primary and secondary engagement metrics across all variants in a single view to understand total performance beyond a single winning metric.
• Validate statistical significance: Confirm whether a winning variation's performance difference reached statistical significance before applying findings to future campaigns.