How long does an A/B test should last?

The duration of an A/B test depends on the kind of test running and on the traffic.

Let’s imagine a test with such a positive modification that the first visitors of the variation make a conversion. You will see very quickly how efficient the test is. But this kind of result is very rare.

Most of the time, the conversion rate does not change a lot right after the launching, but if you keep your test running long enough, the variation will be seen by hundreds of thousands or even millions of visitors, and the results will be relevant. We recommend to use the z-score, a mathematical tool calculating the statistical significance of your result. As you use it, you will learn to detect how many chances a variation has to perform better than another.

Note: if you are using Google Analytics or Kameleoon internal reporting tool, Kameleoon will calculate automatically the significance of your test according to the goals you previously set up in your back-office. For further information, you can read our article about Statistical significance.

Estimating the duration of an A/B test with Kameleoon

You can estimate the duration of your A/B test from Kameleoon editor. To do this, click on the "Finalize your test" button, located on the top right of the Kameleoon editor.

Then, click on the "Estimate duration of the test" button on the bottom of the window.

To have an estimation of the duration, you need to fill in the following information:

  • Daily traffic: traffic in number of daily visitors on the tested page(s).
  • Conversion rate for target goal: conversion rate measured on your website for the goal used as a reference.
  • Required confidence: required confidence to consider the test as significant. For example, a significance level at 95% means your variation has 95% chances to beat the original.
  • Required uplift: the improvement expected between the conversion rate of the reference page and the conversion rate of the variation.

Once you filled in these information, the estimate duration of your A/B test will display on the right part of the window.

Have more questions? Submit a request
Powered by Zendesk