Skip to main content
Calcimator

A/B Test Calculator

Determine statistical significance of A/B test results. Calculate z-score, p-value, conversion lift, and whether your variant beats the control at your chosen confidence level.

Inputs

Results

Statistically Significant?

No

Winner

No clear winner

Control Conv. Rate5%
Variant Conv. Rate6.5%
Relative Lift30%
Z-Score1.44
P-Value0.15
Alpha0.05
How to Use This Calculator
  1. Enter the number of visitors and conversions for both the control and variant groups.
  2. Set your desired confidence level (typically 95%).
  3. Review whether the result is statistically significant and which variant wins.
  4. Check the relative lift percentage to assess practical significance beyond statistical significance.
  5. Only implement the variant if both statistical significance and minimum detectable effect are achieved.

What each input means

Control Visitors
Total number of visitors (or users) in the control group.
Control Conversions
Number of conversions (signups, purchases, clicks) in the control group.
Variant Visitors
Total number of visitors in the variant (test) group.
Variant Conversions
Number of conversions in the variant group.
Confidence Level (%)
The confidence level for determining significance. 95% is standard; 90% is lenient; 99% is strict.

How this is calculated

Worked example, using the default values

  1. Identify Input Parameters
    4 parameters
    Control Visitors = 1000, Control Conversions = 50, Variant Visitors = 1000, Variant Conversions = 65 = 5 input(s) provided
  2. Calculate Statistically Significant?
    No = No
  3. Calculate Winner
    Winner
    No clear winner = No clear winner
  4. Calculate Control Conv. Rate
    Control Conv. Rate
    5 = 5
  5. Calculate Variant Conv. Rate
    Variant Conv. Rate
    6.5 = 6.5

Engine last updated . Checked against 1 independently-derived test — how we verify calculators.

Related Calculators

The questions that sit next to this one — chosen by subject, including calculators filed under a different category.

Data Science

Cohort Analysis Calculator

Analyze user cohort retention over time. Calculate monthly retention rates, churn rates, estimated customer lifetime value (LTV), and total cohort revenue.

Data Science

Confusion Matrix Analyzer

Analyze classification model performance from confusion matrix values. Calculate precision, recall, F1 score, MCC, accuracy, specificity, and more.

Statistics

T-Test Calculator

Calculate the t-statistic, degrees of freedom, and approximate p-value for a one-sample t-test.

Streaming & Content

Thumbnail A/B Test Calculator

Compare click-through rates with statistical significance testing.

Construction & Building Trades

Insulation Calculator

Calculate insulation costs, energy savings, and payback period. Determine the best insulation for your home.

More in Math & Statistics.