Question 823:

Asked on October 22, 2009

Tags: Effect Size , Cohen d , qwkwxajt

1

Answer:

No answer provided yet.An effect size is an important concept in statistics and it's easy to think about but easy to forget about when you're running all those statistical tests. In short, an effect size just tells you how large of a difference there is between two or more means or proportions.

Lets use an educational example. Imagine you were to administer a math test to a set of gifted students and the same math test to a set of remedial math students.  We certainly would expect there to be a difference in scores between groups. This difference is the effect size. Imagine the score is out of 100 and the gifted students score an average score of 90. The remedial math students score on average 45. If you had to describe this difference and knew nothing about statistics, you could say the gifted students scored on average twice as high as the remedial students. Again, this is a simple way of thinking how large of an "effect" there is between remedial and gifted math students. 

To compute an effect size, there are some additional complications. You need to deal with the sample sizes in each group and you need to deal with the variability of the scores. This is easy if both groups have the same fluctuation in scores around the mean and the same sample size. The variation in scores is measured using the standard deviation and it is the average distance values are from the mean.  Let's imagine the standard deviation of the scores on the math test was 15 points for both groups.

One of the more common measures of effect size is Cohen's d. To find the effect size between two groups on some measures (like scores or IQ or height) in the simplest case (when sample sizes and standard deviations are equal) you just subtract the means and divide by the standard deviation. In our example that gets us (90-45)/15 = 3. We would say that there is a 3 standard deviation difference between remedial and gifted students.  That turns out to be a huge difference (see below).

So how is an effect size used in education research?  I can think of two major ways.

1. One way is in describing the magnitude of the difference which we did above.
   
2. The other way is in sample size planning. If you want to know if a new method for teaching math is more effective than a control group and you need to know the sample size, you first need to know how large of a difference you want to be able to detect. If you want to detect larger differences (if they exist at all) then you need a smaller sample size. If you want to detect small differences than you need a larger sample size.You use the effect size (which as we saw is just the number of standard deviations between groups) to determine the sample size.

In general we want to do research on at effect sizes that have an impact. We don't want to spend a ton of money on research only to say that we can improve scores on average by 1/10 of a point--a small effect size.

How large is a large effect size? While it depends on the context, in Psychology, and likely educational research Jacob Cohen in 1988 provided the following guidelines: when comparing two means

1. Small effect size = .3 standard devaiations
2. Medium effect size = .5 standard deviations
3. Lagre effect size >= .8 standard deviations.

The idea is something like a large effect size is something that is noticeable to the naked eye.   For example, the average height of women is around 65 inches (sd 3.5 inches) and the average height of men is 70 inches (sd 4 inches).  If we use the sd of the men we would see the effect size as (70-65)/4 = 1.25, which would be a large effect. That makes sense to me--line up 30 men and 30 women in a room and you should notice that the men are noticeably taller than the women.

For more information on Cohen's d and the effect size see the wikipedia entry.