: How much traffic is needed to effectively conduct split testing? A friend who's asking me for advice about getting more customers from his website (SEO, conversion rate, some other stuff), thinks

@Pierce454

You can perform split testing with any volume of website traffic, but in order to identify whether any increases in conversion rate are statistically significant (as opposed to being down to chance) you would need to run the test for longer than you would for a site with higher traffic.

There are numerous sites that will calculate whether the results of your split testing are significant or not. A Google search for split test calculator returns many sites that perform the same service.

Visual Website Optimizer's A/B Split and Multivariate Test Duration Calculator will also calculate how many days you need to run a test for based on existing and desired conversion rates and visitor numbers.

Vote Up Vote Down

@Gretchen104

You can find the reason at the following URL: en.wikipedia.org/wiki/Law_of_large_numbers.
However, it is difficult to know exactly what numbers you need. From experience, I do not feel comfortable with a split test unless there are a few thousands entries (5000+).

Then it depends of course by the nature of your website, and on what you want to test. The smaller the change, the higher number of entries you'd need to appreciate the difference.

When you have such a small traffic (100 a day), I believe one of the best solutions is to ask a few friends with different background (reflecting the type of users distribution that would normally visit your site), as described in this page en.wikipedia.org/wiki/Usability_testing#How_many_users_to_test.3F. Split testing is a very useful tool, but you need large numbers to make it work reliably.

Vote Up Vote Down

@Annie201

Well there is no magic number of visits needed when it comes to running a split test or multivariate test. It really depends on what you are testing, and what you have selected as a conversion. If you set a sale to be a conversion and you don't have many to begin with you'll be waiting a long time to see stable results.

What you should be doing is focusing on a page which has the most visits. Your home page most likely, maybe category pages. Maybe even the combined pageviews of all your product pages is the highest number of all. So first figure out which page you're going to test. Then ask yourself what is the next logical step a visitor needs to take to start their checkout process. This is assuming eCommerce but can be applied to any site. So lets say from a product page the next step is clicking add to cart. So you set your conversion page to be the View cart page or Check out page whichever is seen after adding a product to the cart.

Now you'll create a test for content or elements on the product page. Since far more people will be adding products to their cart and starting the check out process you'll get results much sooner which will tell you what is it that's stopping or helping visitors decide to add a product to their cart. Once you have the results from that first test you can start another or begin a test on your view cart page. And little by little fine tune your websites flow to funnel your visitors to the check out process.

You also do not have to wait until Google tells you the test is finished. Give it some time for the up's and down's to settle in and eventually in the chart you'll see two steady lines indicating your test version vs original and one will be a clear winner steadily week after week. Once you see that end it and start another.

Vote Up Vote Down