Experimentation
A/B test
Also calledSplit test, A/B testing, Online controlled experiment
An experiment that shows two versions of something to randomly split groups of users, to measure which one performs better.
Visitors are randomly assigned to version A (usually the current one) or version B (the change). Because the split is random, any meaningful difference in results can be credited to the change rather than to who happened to see it.
A good test is planned before it starts: one main metric, a few guardrail metrics that must not get worse, the sample size needed, and how long it will run.
Common mistake
Checking the result every day and stopping the moment it looks significant. Peeking inflates false positives; decide the sample size up front and wait for it.