The new A/B Testing feature lets you send multiple versions of a broadcast to segments of your audience, see which one performs best, and automatically (or manually) send the winning version to the remaining audience - all without building and sending separate broadcasts by hand.

Switchboard tracks the performance for each variant and the winner separately, so you can see exactly how each version did.

This feature is currently in beta; core functionality is stable, but some capabilities (like per-version media attachment) are still actively being built out. Please share any feedback or unexpected behavior with support@oneswitchboard.

How to set up A/B Tests for Broadcasts

1. Create your test versions

• Start a new broadcast (or open a draft).

• Check the box to make the broadcast an A/B test or click + New Version (in the upper righthand corner by the broadcast editor) to convert it into an A/B test. You can add up to 7 versions.

• Edit each version's message content and links.

2. Set your test audience and split

• Attach a List or Saved Search.

• Split the audience across your variants by percentage or count. The remainder is reserved for your winner.

  • Note: the audience split will be approximate if your audience includes any opt outs or likely to skip numbers.

3. Choose how the winner will be determined

• Automatic Send - Switchboard evaluates your winner criteria at the evaluation date/time and automatically sends the winning version to the rest of your audience — no action needed from you.

• Manually Select Winner - Switchboard pauses the broadcast after the test versions send. You can then:

  • Accept the suggested winner (if you defined winner criteria) by clicking "Resume", or

  • Manually select a different winner yourself.

4. Set your winner criteria

• Choose your winning metric: ROI and/or Minimum donation count.

• Set an evaluation date/time — this is when Switchboard checks the test results and determines a winner.

5. Add Expansions to the winner

• Scroll down to "Manage Expansions" to add Pause/Unpause Automatic Expansion steps to the winner

A/B Testing + Send Ordering

Send by Recipient Time Zone

We don't recommend using Recipient Time Zone send ordering in conjunction with A/B testing as it can produce unexpected results.

• What works:

  • Your test versions will respect recipient time zone ordering, just like a normal time-zone-ordered broadcast.

• What doesn't work the same way:

  • In most cases, the test winner will not respect time zone ordering since the originally scheduled time will likely be in the past by the time the test is complete. If you do use Scheduling by Recipient Time Zone + A/B test, treat the winner send as happening all at once rather than staggered by zone.

Send by Custom Score

• You can use Custom Score send ordering in conjunction with A/B tests. Your sorted audience will be split in two. The first group is divided randomly across your test versions and the second group will receive the Winner.

FAQs

When will I receive notifications about my test?

• Regardless of whether you select Auto Send or Manually Select the Winner, you will receive an email notifying you once the test versions have finished sending.

• If you select Auto Send or Manually Select the Winner and include an evaluation date and ROI threshold, you will also receive an email notifying you of the test results.

Can I edit the winning version before it sends?

• If you have selected to Automatically Send the Winner, the winning version content (including refcode) will be copied over as-is.

• If you have opted to Manually Select the Winner, you can manually edit the content of the winning version (including to edit the refcode) or even manually override to select a different winning version to send.

If you prefer to maintain more control over your A/B test, choose the Manually Select the Winner option.

Can I attach different media files to each test version?

• Not yet — one media attachment applies across all test versions in this beta. Distinct media per test version is planned and coming soon.

Where can I see how my test performed after the fact?

• Check your broadcast's stats page. Test version and winner performance stats are tracked separately. Reference the stats as close to your evaluation date/time as possible for the most accurate picture.

What if my test results are a tie or nobody donated?

• The test is marked inconclusive, and you'll need to manually select a winner yourself — this applies whether you chose Automatic Send or Manual Select.

Have Feedback?

This feature is still being refined based on beta feedback. If you run into unexpected behavior, have suggestions, or want to share a screen recording of something confusing, please reach out to support@oneswitchboard.com — we'd love to hear from you.