A/B testing (split testing)
A/B testing is comparing two versions of an email — differing in one variable like the subject line, content or send time — by sending each to a sample and measuring which performs better before rolling the winner out.
How A/B testing works
You create two variants that differ in exactly one element, then send each to a random, comparable slice of your audience. After enough responses accumulate, you compare a chosen metric — opens, clicks or conversions — and declare a winner. Many platforms can then send the winning version to the remaining contacts automatically.
The discipline is changing one thing at a time. Test the subject line and the CTA and the image at once, and a lift tells you nothing about which change caused it.
What to test
- Subject line and preview text — the biggest lever on open rate
- Call-to-action wording, colour and placement — drives clicks
- Send time and day — matches your audience's habits
- From name — affects trust and opens
- Content layout, length and imagery — influences clicks and conversions
Running a valid test
- Change one variable per test so the result is attributable
- Use samples large enough to reach a reliable result, not a coin flip
- Pick the metric that matches the goal — conversions over opens for revenue
- Give the test enough time to gather responses before deciding
- Document what won and feed it into the next test — testing compounds
Frequently asked questions
How big does my list need to be to A/B test?
Large enough that each variant gets a meaningful number of responses — otherwise the 'winner' is noise. Small lists can still test high-frequency automated flows, where samples accumulate over time.
What should I test first?
The subject line, because it has the largest effect on open rate and is easy to isolate. Once opens are optimised, move to CTAs and content that drive clicks and conversions.
Why did my A/B test give a misleading result?
Usually too small a sample, testing several changes at once, or ending too early. Change one variable, use adequate samples, and let the test run its course.
Start sending in minutes
Create a free account, connect your domain and reach your audience across every channel — no credit card needed.
Free plan forever · No credit card required · Set up in minutes