: Weird Results A/B Test in Google Website Optimizer I set up a test in Google Website Optimizer that has a 3 variations - original (A), B, and C. In order to further validate the results of

@Shelton105

Not enough experiments or divergence in the results to be statistically significant yet. Try doing a Google search on AB Testing confidence and work out what sort of confidence level you're at with your experiment. This will give you a much better idea of whether your results are significant or whether you should just let the experiment run longer.

Speaking from personal experience I cannot tell you how many times I've tested an altered landing page layout that instantly showed a huge improvement in an AB test in the first day or two. But then as the amount of data increases my "improvement" turned out to be a pipe dream/

Vote Up Vote Down

@Jessie594

If C was displayed to visitors even a small percentage over A it would have a higher conversion rate. With now 3 variations you cannot distribute your traffic evenly, so there's probably always one variation with slightly more views than another.

I often use the multivariate tool with a single variation for A/B tests on dynamic pages.

6 days is also a short amount of time even with a few hundred visits. But that also depends on what your conversion is considered. Leaving the tests to run a month or more is ideal to start to see a steady pattern. In the beginning of tests it's common to see variations bouncing up and down with conversions. After a while the lines steady out and you'll see a clear winner.

I suggest not adding a variation that is the exact same as your test / original page

Vote Up Vote Down