While you may hear the margin of error reported on TV or in the paper, the confidence level is a crucial component that's often left out. Quite simply, the confidence level represents the likelihood that another sample will provide the same results. It is the percent likelihood statement that accompanies the width of the confidence interval. It is often set to the 95% level by convention but can be adjusted (see the confidence level in the figure below).
A confidence level of 95% means that 95 out of 100 times the sample percentages will fall within the confidence intervals. Or 5 times out of 100 the percentages will NOT fall within the confidence intervals. So if the Star Tribune took the same poll 100 times with a margin of error of 6% at 95% confidence, we'd expect that about 5 of those polls would show Jim Bean to have more than 61% of the vote or less than 49% of the vote.
**Why 95%?** 95% is the most frequent value of the confidence level and it is set that way mostly by convention (5% seemed like a reasonable amount of risk I suppose). You would want to lower it to 90% or 85% or raise it to 99% depending on the impact of being wrong. In other words, if you were betting a lot of money on where the 100 sample results would fall, then you'd want to use a 99% confidence level, not an 80% level (unless you don't mind a 20% chance of having to pay up). But, there is a price to pay for having a more precise estimate, and that's a wider confidence interval. |