A/B test significance calculator
Enter the recipients and conversions for each variant to see the conversion rates, the uplift between them, and the confidence level from a two-proportion z-test — so you know if the winner is real or just noise.
6.00%
7.50%
+25.0%
Confidence: 94.1% · Variant B is ahead.
What significance means here
When one variant beats another, you need to know whether the gap reflects a real difference or random chance. This calculator runs a two-proportion z-test: it compares the conversion rates and reports a confidence level. The common threshold is 95% — below that, the result could easily reverse with more data.
A 'conversion' can be any binary outcome: an open, a click, a purchase. Enter the count of recipients (or visitors) and how many converted for each variant.
- Conversion rate — conversions ÷ recipients, for each variant
- Relative uplift — how much B beats (or trails) A, in percent
- Confidence — the probability the difference is not due to chance
Running trustworthy tests
- Decide your sample size before starting — don't stop the moment it looks good
- Test one variable at a time so you know what caused the change
- Wait for significance; small samples swing wildly early on
- Even at 95%, one in twenty 'wins' is a fluke — confirm big changes
Frequently asked questions
What confidence level should I aim for?
95% is the standard for marketing tests — a 1-in-20 chance the result is noise. For high-stakes decisions, wait for 99%.
My test isn't significant — what now?
Keep it running to gather more data, or accept the variants perform similarly. Stopping early at a flattering moment is the most common A/B testing mistake.
Does this work for open and click tests?
Yes. Any binary outcome works — enter recipients and the count who opened, clicked or purchased for each variant.
Start sending in minutes
Create a free account, connect your domain and reach your audience across every channel — no credit card needed.
Free plan forever · No credit card required · Set up in minutes