: Sample size and statistical significance in Google Analytics I have been asked to compile a report into dropout rates during checkout for a global webstore I have used a sample size over one

@Cugini213

(I'm not an expert in statistics.)

One month's data is almost certainly not representative, for a couple of reasons.


Most sites have seasonal trends (both monthly and weekly). Picking a really light month or a really heavy month will skew your results.
You'll probably need a sample size of more than 1/12 to get results worth betting business on.


It would seem to make more sense to take a random sample of 'n' days within a single year, where 'n' is chosen to give you the confidence level and margin of error you feel you can justify.

There are online tools that can help you choose a sample size. I linked to the first one I found. I don't have any specific experience with it or the company that hosts it.


Am I right in thinking that I need to compare the standard deviation
of my monthly sample to the yearly sample and ensure that the
deviation is under a certain %age?


I don't think so, but I could easily be wrong. The standard deviation of a very small sample might match your population standard deviation without the sample size being representative.

I need to brush up on my stats.

Vote Up Vote Down

@Hamaas447

Statistically significant is a terms used in Maths that means your data is indicative of a certain event and that you have used enough data to prove it.

For example, if 5 people came into your shop and 1 of them bought something, you can't then advertise that 1 in 5 people spend money in your shop.

When I was doing my A-levels, we had lookup tables for different distributions and sample sizes where with some calculations you could calculate how significant your results were. Unfortunately my brain gives out at this point trying to remember exactly how... A Google search should throw up more info.

Vote Up Vote Down

@LarsenBagley505

There's usually traffic patterns for every eCommerce site. Peaking around Nov through Dec. Halloween peaks in Oct and is dead the rest of the year. Google search traffic varies month to month. Login to adwords go to the keywords tool and export to CSV and look at the search volume for some of your keywords you'll see what Google shows in the keyword tool is the average of the last year. While the CSV will show you how some months search volume goes up quite a bit compared to others.

How would you pick a single month of data to analyze when traffic and search volume can vary from one month to another?

If Google is running slow when you're trying to generate a yearly report you should drill down and create very specific custom reports for 1 years time. I've generated reports for 1 year with comparison data of the previous year and got them pretty quickly. I would imagine that your bounce rate from the checkout pages is much less data than gathering visits to your home page.

I would do what it takes to get not 1 but two years of data to compare and show them what's changed in bounce rates over the last year.

Vote Up Vote Down

@Goswami781

I would show that the traffic is not seasonal, so there should not be much difference comparing any month with each other (assuming no significant changes in the website/traffic source has been done).

Standard deviation would be one signal, but I think any descriptive statistics analysis would do it. However you should not base your demonstration solely on numbers, but behaviour as well (even if only main traits, then going into depth for the month you analysed).

Vote Up Vote Down