When we conduct a piece of quantitative research, we are inevitably attempting to answer a hypothesis that we have set. Since hypothesis testing has many facets, let’s look at an example.
Two statistics lecturers, Sarah and Mike, think that they use the best method to teach their students. Each lecturer has 50 statistics students that are studying a graduate degree in management. In Sarah’s class, students have to attend one lecture and one seminar class every week, whilst in Mike’s class students only have to attend one lecture. Sarah thinks that seminars, in addition to lectures, are an important teaching method in statistics, whilst Mike believes that lectures are sufficient by themselves and thinks that students are better off solving problems by themselves in their own time. This is the first year that Sarah has given seminars, but since they take up a lot of her time, she wants to make sure that she is not wasting her time and that seminars improve her students’ performance.
Whilst all pieces of quantitative research have some dilemma, issue or problem that they are trying to investigate, the focus in hypothesis testing is to find ways to structure these in such a way that we can test them effectively. Typically, it is important to:
|1.||Define the research hypothesis and set the parameters for the study.|
|2.||Set out the null and alternative hypothesis (or more than one hypothesis; in other words, a number of hypotheses).|
|3.||Explain how you are going to operationalise (that is, measure or operationally define) what you are studying and set out the variables to be studied.|
|4.||Set the significance level.|
|5.||Make a one- or two-tailed prediction.|
|6.||Determine whether the distribution that you are studying is normal (this has implications for the types of statistical tests that you can run on your data).|
|7.||Select an appropriate statistical test based on the variables you have defined and whether the distribution is normal or not.|
|8.||Run the statistical tests on your data and interpret the output.|
|9.||Accept or reject the null hypothesis.|
Whilst there are some variations on this structure, it is adopted by most thorough quantitative research studies. Throughout this guide we discuss this hypothesis testing process using our example of the two statistics lecturers, Sarah and Mike, and their alternative teaching methods. We focus on the first fives steps in the process, as well as the decision to either accept or reject the null hypothesis. A discussion of normality, selecting statistical tests, and running these statistical tests is discussed in the statistical guide, Selecting Statistical Tests. At the end, we highlight some of the other statistical guides that we think you would benefit from reading before going on to design your own quantitative piece of research.
This study aims to examine the effect that two different teaching methods – providing lectures and seminar classes (Sarah) and providing lectures by themselves (Mike) – had on the performance of Sarah’s 50 students and Mike’s 50 students. By establishing that we are not only interested in these 100 students, we set the parameters for the study. This is important because if we were interested in the effect that these teaching methods had on students’ performance in general, there would be wider sampling implications (see the statistical guide, Sampling, for more information).
Whilst Mike is sceptical about the effectiveness of seminars, Sarah clearly believes that giving seminars, in addition to lectures, helps her students do better than those in Mike’s class. This leads us to the following research hypothesis:
|Research Hypothesis:||When students attend seminar classes, in addition to lectures, their performance increases.|
Whilst only providing a research hypothesis like the above is sometimes adequate, it is good practice in quantitative research to re-state this as a null and alternative hypothesis.
|Null Hypotheses (Ho):||Undertaking seminar classes has no effect on students’ performance.|
|Alternative Hypothesis (Ha):||Undertaking seminar class has a positive effect on students’ performance.|
The null hypothesis predicts that the distributions that we are comparing are the same. This brings us back to the core of hypothesis testing; comparing distributions (see the statistical guide, Frequency Distributions, for more information on distributions). In this study, there are two distributions that we are comparing.
|Distribution 1 (seminar)||The distribution of exam marks for the 50 students in Sarah’s class that attended both lectures and seminars.|
|Distribution 2 (lecture only)||The distribution of exam marks for the 50 students in Mike’s class that attended only lectures.|
If the two distributions are the same this would mean that the addition of seminars to lectures as a teaching method did not have an effect on students’ performance and we would accept the null hypothesis. Alternatively, if there was a difference in the distributions and this difference was statistically significant, we would reject the null hypothesis. The question then arises: Do we accept the alternative hypothesis?
Before we answer this question and three related concepts (the alternative hypothesis, one- and two-tailed predictions, and statistical significance), it is worth addressing the issue of operationally defining our study.