Pre-test Planning
Determine how many visitors you need and how long your test should run to detect a material lift with statistical confidence.
Plan Inputs
The smallest relative lift you want to be able to detect.
Adding variants increases required sample size (Bonferroni adjustment).
Prescriptive Advice
At your current traffic, this test would take 63 days. That's enough time for the surface itself to change underneath you (other releases, seasonality, traffic mix shifts). Consider: (a) targeting a larger MDE, (b) increasing traffic allocation to the test, or (c) skipping the A/B test and using qualitative research instead.
To detect a 10% lift on a 5.00% baseline, you need 62,462 total visitors.
How many people you need
Smaller lifts need far more visitors: the scale goes up by ten at each line. The dashed line is four weeks of your traffic.
With four weeks of your traffic, the smallest lift you could reliably detect is 15.1%. That's bigger than the lift you planned for, so plan to run longer or test a bolder change.
Pro Tip
If the required sample size is too high, consider increasing your MDE. Smaller effects are harder to see and require exponentially more traffic. Aim for a 14-day minimum duration regardless of what the math says to account for full weekly cycles.