Laerd Statistics LoginCookies & Privacy

Hypothesis Testing (cont...)

A one-tailed prediction indicates that you believe that your distribution (the "seminar" distribution) is either higher up (diagram A) or lower down (diagram B) the scale (in this case, exam marks) compared with the alternative distribution (the "lectures only" distribution).

Alternately, a two-tailed prediction means that we do not make a choice over the direction that our distribution moves. Rather, it simply implies that our distribution could be either higher up or lower down the scale. If Sarah had made a two-tailed prediction, our alternative hypothesis might have been:

Alternative Hypothesis (Ha): Undertaking seminar class has an effect on students’ performance.

In other words, we simply take out the word "positive", which implies the direction of our effect. In our example, making a two-tailed prediction may seem strange, which would be a fair comment. After all, it would be logical to expect that “extra” tuition (going to seminar classes as well as lectures) would either have a positive effect on students’ performance or no effect at all, but certainly not a negative effect. Therefore, the two distributions would either be the same or the seminar distribution would move higher up the scale. Nonetheless, there are cases when we do not know what the effect might be. For example, rather than using seminars in addition to teaching, Sarah could have used an untested, experimental teaching method. Since this teaching method was experimental, we may not be able to assume whether it would have a positive or negative effect on students’ performance, or simply no effect at all.

Significance levels

Whilst we are close to being able to either accept or reject the null hypothesis, and if we reject it, either accept or reject the alternative hypothesis, rigour requires that we first set the significance level for our study.

Statistical significance is about probability. It asks the question: What is the probability that a score could have arisen by chance? In terms of our two distributions, statistically analysing the differences in the distribution may, for example, suggest that there are no differences in the two distributions; hence, we should accept the null hypothesis. However, how confident are we that there really are no differences between the distributions?

Join the 1,000s of students, academics and professionals who rely on Laerd Statistics. TAKE THE TOUR

Typically, if there was a 5% or less chance (5 times in 100 or less) that a score from the focal distribution (the "seminar distribution") could not have come from the comparison distribution (the "lectures only" distribution), we would accept the null hypothesis. Alternately, if the chance was greater than 5% (6 times in 100 or more) we would reject the null hypothesis. We do not reject the null hypothesis because our statistical analysis did not show that the two distributions were the same. We reject it because at a significance level of 0.05 (i.e., 5% or less chance), we could not be confident enough that this result did not simply happen by chance.

Whilst there is relatively little justification why a significance level of 0.05 is used rather than 0.04 or 0.10, for example, it is widely accepted in academic research. However, if we want to be particularly confident in our results, we set a more stringent level of 0.01 (a 1% chance or less; 1 in 100 chance or less).

Accepting or rejecting the null hypothesis

Therefore, let’s return finally to the question of whether we (a) reject or accept the null hypothesis; and (b) if we reject the null hypothesis, do we accept the alternative hypothesis?

If our statistical analysis shows that the two distributions are the same at the significance level (either 0.05 or 0.01) that we have set, we simply accept the null hypothesis. Alternatively, if the two distributions are different, we need to either accept or reject the alternative hypothesis. This will depend on whether we made a one- or two-tailed prediction (see below).

1 2 3 4