Thumbnail A/B Test Calculator

The Z-Score measures how many standard errors apart the two variants' click-through rates are, using a pooled two-proportion z-test. A Z-Score above roughly 1.96 means the gap is unlikely to be due to random chance at the conventional 95% confidence level, which is exactly the threshold Significant at 95% checks against.

CTR (A) is calculated purely from Clicks (A) divided by Impressions (A) -- it is Variant A's own click-through rate and has no dependency on Variant B's traffic or clicks. Variant B's numbers only enter the calculation once you look at Lift, Z-Score, and Significant at 95%, which compare the two variants directly.

Down. CTR (A) divides Clicks (A) by Impressions (A), so adding impressions without adding clicks always spreads the same click count over a larger base and lowers the rate. Raising Clicks (A) with impressions held steady has the opposite effect and always raises CTR (A).

Yes, if the impression counts are large enough. Statistical significance depends on both the size of the CTR gap and the sample size behind it -- a tiny 0.2 percentage-point difference can still clear the 95% threshold if it's measured across millions of impressions, because the standard error shrinks as sample size grows.

Min Sample Each is a rough estimate of how many impressions each variant would need to reliably detect a CTR difference of the size you're currently observing, at 95% confidence and 80% statistical power. It's a planning guide for how long to keep collecting data before drawing conclusions, not a hard requirement.