: Why is Google Analytics experiment not being shown equally? I have set-up and simple A/B test with 100% of my traffic, everything seems to be running fine except the original page and the page

@Angela700

Google content experiments uses a technique called "multi-armed bandit" methodology.
support.google.com/analytics/answer/2844870?hl=en-GB&ref_topic=2844866
Basically, it seeks to speed up the test and increase conversions throughout the test by sending more traffic to the 'winning' pages throughout the experiment.

If version 2 performs better on day one then it will receive more traffic the following day and the cumulatively afterwards. The original page still gets traffic so if that then performs well one day it will get a larger share of traffic the following day.

There are lots of thoughts about multi-armed bandit testing vs 50:50 testing but basically, don't worry as with google content experiments there isn't anything you can do about it.

Vote Up Vote Down

@Jessie594

Tricky question, the default settings for Google's content experiements are a 2 week duration with a confidence level of 95%. If you got low traffic volume, these confidence levels may not be reached, because of differences in user behaviour on different days of the week (weekdays vs. weekends).

Even if you implement a full 100% traffic split, the testing suite may shift traffic volumes to different variants trying to find confidence levels and thus be able to declare a winner.

TL;DR
A/B testing does not route the first visitor to variant A and the second to variant B. Instead it's trying to segment types of traffic to be able to compare them. I have never seen a A/B test where traffic was split exactly 50% (between two variants), no matter how long the experiement was running.

Vote Up Vote Down