There are three areas that affect the width of a confidence interval. - Sample Size
- Confidence Level
- Population Variability
As the sample size decreases, the confidence intervals get wider. As the confidence level increases the confidence intervals also get wider. Guess what else? As the variability of the population you're sampling from increases the confidence interval of your sample gets wider. So what is population variability? It's how much the individual data points differ from each other in the whole population.
While populations are usually very large (like the millions of people in a country or thousands of patients at a hospital) they can also be much smaller. Let's imagine there's just 20 users at a small company that use an intranet application. Now, imagine if you had time to test all 20 on two verisons of the same intranet applications (10 on each version). You'd have two populations of 10 users. One population using version 1 and one population using version 2 (even though these folks work with each other at the same company, they are in different statistical populations since we're measuring them on different versions of an intranet). Let's say that on average both sets of 10 users took exactly 60 seconds to complete the task on each version of the intranet application (see below). Which set of times has more variability ? | Version 1 Avg. Time: 60 Seconds (10 Users) | Version 2 Avg. Time: 60 Seconds (10 Users) | | 95 48 71 50 74 40 25 64 51 82 | 150 21 100 40 33 74 12 133 21 16 | Now, which set of times has more variability ? (Click the Start Task buttons to see the variability)
Notice how the sample from Version 2 has a more sporadic distribution of times? It seems like the user times are all over the place compared to the Version 1 times. Often when data is visually depicted, you can notice the difference in variability rather quickly. |