A mixed ANOVA compares the mean differences between groups that have been split on two "factors" (also known as independent variables), where one factor is a "within-subjects" factor and the other factor is a "between-subjects" factor. For example, a mixed ANOVA is often used in studies where you have measured a dependent variable (e.g., "back pain" or "salary") over two or more time points or when all subjects have undergone two or more conditions (i.e., where "time" or "conditions" are your "within-subjects" factor), but also when your subjects have been assigned into two or more separate groups (e.g., based on some characteristic, such as subjects' "gender" or "educational level", or when they have undergone different interventions). These groups form your "between-subjects" factor. The primary purpose of a mixed ANOVA is to understand if there is an interaction between these two factors on the dependent variable. Before discussing this further, take a look at the examples below, which illustrate the three more common types of study design where a mixed ANOVA is used:
Study Design #1
Your within-subjects factor is time.
Your between-subjects factor consists of conditions (also known as treatments).
Imagine that a health researcher wants to help suffers of chronic back pain reduce their pain levels. The researcher wants to find out whether one of two different treatments is more effective at reducing pain levels over time. Therefore, the dependent variable is "back pain", whilst the within-subjects factor is "time" and the between-subjects factor is "conditions". More specifically, the two different treatments, which are known as "conditions", are a "massage programme" (treatment A) and "acupuncture programme" (treatment B). These two treatments reflect the two groups of the "between-subjects" factor.
In total, 60 participants take part in the experiment. Of these 60 participants, 30 are randomly assigned to undergo treatment A (the massage programme) and the other 30 receive treatment B (the acupuncture programme). Both treatment programmes last 8 weeks. Over this 8 week period, back pain is measured at three time points, which represents the three groups of the "within-subjects" factor, "time" (i.e., back pain is measured "at the beginning of the programme" [time point #1], "midway through the programme" [time point #2] and "at the end of the programme" [time point #3]).
At the end of the experiment, the researcher uses a mixed ANOVA to determine whether any change in back pain (i.e., the dependent variable) is the result of the interaction between the type of treatment (i.e., the massage programme or acupuncture programme; that is, the "conditions", which is the "between-subjects" factor) and "time" (i.e., the within-subjects factor, consisting of three time points). If there is no interaction, follow-up tests can still be performed to determine whether any change in back pain was simply due to one of the factors (i.e., conditions or time).
Study Design #2
Your within-subjects factor is time.
Your between-subjects factor is a characteristic of your sample.
Imagine that a researcher wants to determine whether stress levels amongst young, middle-aged and older people change the longer they are unemployed, as well as understanding whether there is an interaction between age group and unemployment length on stress levels. Therefore, the dependent variable is "stress level", whilst the "within-subjects" factor is "time" and the "between-subjects" factor is "age group".
In total, 60 participants take part in the experiment, which are divided into three groups with 20 participants in each group, which reflects the between-subjects factor, "age group" (i.e., the 3 groups are "young", "middle-aged" and "older" unemployed people). The dependent variable, "stress level", is subsequently measured over four time points, which reflects the within-subjects factor, "time" (i.e., stress levels are measured "on the first day the participants are unemployed" [time point #1], "after one month of unemployment" [time point #2], "after three months of unemployment" [time point #3] and "after six months of unemployment" [time point #4]).
At the end of the experiment, the researcher uses a mixed ANOVA to determine whether any change in stress level (i.e., the dependent variable) is the result of the interaction between age group (i.e., whether participants are "young", "middle-aged" or "older"; the "between-subjects" factor) and "time" (i.e., the length that the groups of people are unemployed; the "within-subjects" factor). If there is no interaction, follow-up tests can still be performed to determine whether any change in stress levels was simply due to one of the factors (i.e., time or age group).
Study Design #3
Your within-subjects factor consists of conditions (also known as treatments).
Your between-subjects factor is a characteristic of your sample.
Imagine that a psychologist wants to determine the effect of exercise intensity on depression, taking into account differences in gender. Therefore, the dependent variable is "depression" (measured using a depression index that results in a depression score on a continuous scale), whilst the "within-subjects" factor consists of "conditions" (i.e., 3 types of "exercise intensity": "high", "medium" and "low") and the "between-subjects" factor is a "characteristic" of your sample (i.e., the between-subjects factor, "gender", which consists of "males" and "females"). More specifically, these three different "conditions" (also known as "treatments") are a "high intensity exercise programme" (treatment A), "medium intensity exercise programme" (treatment B) and "low intensity exercise programme" (treatment C). Each of these three treatments (i.e., treatment A, treatment B and treatment C) reflect the three groups of the "within-subjects" factor, "exercise intensity".
In total, 45 participants take part in the experiment. Since "exercise intensity" is the "within-subjects" factor, this means that all 45 participants have to undergo all three treatments: the "high intensity exercise programme" (treatment A), "medium intensity exercise programme" (treatment B) and "low intensity exercise programme" (treatment C). Each treatment lasts 4 weeks. However, the order in which participants receive each treatment differs, with the 45 participants being randomly split into three groups: (a) 15 participants first undergo treatment A (the "high intensity exercise programme"), followed by treatment B (the "medium intensity exercise programme"), and finally treatment C (the "low intensity exercise programme"); (b) another 15 participants start with treatment B, followed by treatment C, and finishing with treatment A; and (c) the final group of 15 participants start with treatment C, followed by treatment A, and finally, treatment B. This is known as counterbalancing and helps to reduce the bias that could result from the order in which the treatments are provided (although you may not have done this in your research).
At the end of the experiment, the psychologist uses a mixed ANOVA to determine whether any change in depression (i.e., the dependent variable) is the result of the interaction between exercise intensity (i.e., the "conditions/treatments", which is the within-subjects factor) and gender (i.e., a "characteristic" of the sample, which acts as the between-subjects factor). If there is no interaction, follow-up tests can still be performed to determine whether any change in depression was simply due to one of the factors (i.e., exercise intensity or gender).
As mentioned above, the primary purpose of a mixed ANOVA is to understand if there is an interaction between your within-subjects factor and between-subjects factor on the dependent variable. Once you have established whether there is a statistically significant interaction, there are a number of different approaches to following up the result. In particular, it is important to realize that the mixed ANOVA is an omnibus test statistic and cannot tell you which specific groups within each factor were significantly different from each other. For example, if one of your factors (e.g., "time") has three groups (e.g., the three groups are your three time points: "time point 1", "time point 2" and "time point 3"), the mixed ANOVA result cannot tell you whether the values on the dependent variable were different for one group (e.g., "Time point 1") compared with another group (e.g., "Time point 2"). It only tells you that at least two of the three groups were different. Since you may have three, four, five or more groups in your study design, as well as two factors, determining which of these groups differ from each other is important. You can do this using post hoc tests, which we discuss later in this guide. In addition, where statistically significant interactions are found, you need to determine whether there are any "simple main effects", and if there are, what these effects are (again, we discuss this later in our guide).
If you are unsure whether a mixed ANOVA is appropriate, you may also want to consider how it differs from a two-way repeated measures ANOVA. Both the mixed ANOVA and two-way repeated measures ANOVA involve two factors, as well as a desire to understand whether there is an interaction between these two factors on the dependent variable. However, the fundamental difference is that a two-way repeated measures ANOVA has two "within-subjects" factors, whereas a mixed ANOVA has only one "within-subjects" factor because the other factor is a "between-subjects" factor. Therefore, in a two-way repeated measures ANOVA, all subjects undergo all conditions (e.g., if the study has two conditions – a control and a treatment – all subjects take part in both the control and the treatment). Therefore, unlike the mixed ANOVA, subjects are not separated into different groups based on some "between-subjects" factor (e.g., a characteristic such as subjects' "gender" or "educational level", or so that they only receive one "condition": either the control or the treatment). Therefore, if you think that the mixed ANOVA is not the test you are looking for, you may want to consider a two-way repeated measures ANOVA. Alternately, if neither of these are appropriate, you can use our Statistical Test Selector, which is part of our enhanced content, to determine which test is appropriate for your study design.
In this "quick start" guide, we show you how to carry out a mixed ANOVA with post hoc tests using SPSS Statistics, as well as the steps you will need to go through to interpret the results from this test. However, before we introduce you to this procedure, you need to understand the different assumptions that your data must meet in order for a mixed ANOVA to give you a valid result. We discuss these assumptions next.
When you choose to analyse your data using a mixed ANOVA, much of the process involves checking to make sure that the data you want to analyse can actually be analysed using a mixed ANOVA. You need to do this because it is only appropriate to use a mixed ANOVA if your data "passes" seven assumptions that are required for a mixed ANOVA to give you a valid result. In practice, checking for these assumptions requires you to use SPSS Statistics to carry out a few more tests, as well as think a little bit more about your data. Whilst it is not a difficult task, it will take up most of your time when carrying out a mixed ANOVA.
Before we introduce you to these seven assumptions, do not be surprised if, when analysing your own data using SPSS Statistics, one or more of these assumptions is violated (i.e., not met). This is not uncommon when working with real-world data rather than textbook examples. However, even when your data fails certain assumptions, there is often a solution to try and overcome this. First, let’s take a look at these seven assumptions:
You can check assumptions #4, #5, #6 and #7 using SPSS Statistics. Just remember that if you do not run the statistical tests on these assumptions correctly, the results you get when running a mixed ANOVA might not be valid. This is why we dedicate a number of sections in our enhanced guide to help you get this right. You can find out about our enhanced content as a whole here, or more specifically, learn how we help with testing assumptions here.
In the section, Procedure, we illustrate the SPSS Statistics procedure that you can use to carry out a mixed ANOVA on your data. First, we introduce the example that is used in this guide.
A researcher wanted to discover whether the intensity of an exercise-training programme, but with equal calorific expenditure, had an effect on cholesterol concentration over a six-month period. Therefore, the dependent variable was "cholesterol concentration", the within-subjects factor was "time" and the between-subjects factor was the "conditions" (N.B., each of these variables are explained further below).
To answer this, 60 participants were recruited to take part in the experiment, randomly split into three each groups of 20 participants. Each of these three groups of 20 participants received a different "condition": in one group, participants did not change their current sedentary lifestyle (i.e., this was Group #1, also called the "control" group); in another group, participants underwent a low-intensity exercise-training programme that expended 1000 kCal per week (i.e., this was Group #2, also called "treatment A"); the final group underwent a high-intensity exercise-training programme that also expended 1000 kCal per week, but therefore exercised for less total time (i.e., this was Group #3, also called "treatment B"). All of the conditions (i.e., the control, treatment A and treatment B) lasted six months. During this period, the dependent variable, "cholesterol concentration", was measured three times: "at the beginning of the experiment" (time point #1), "mid-way through the six months" (time point #2) and "at the end of the experiment" (time point #3). These three time points (i.e., time point #1, time point #2 and time point #3) represent the three groups of the within-subjects factor, "time".
In this example, there are three variables: (1) the dependent variable, cholesterol, which is the cholesterol concentration (in mmol/L); (2) the between-subjects factor, group, which has three categories: "Control" (control group), "Int_1" (treatment A) and "Int_2" (treatment B); and (3) the within-subjects factor, time, which has three categories: "pre", "mid" and "post".
Participants' cholesterol concentration was recorded in the variable pre for pre-intervention, mid for mid-way through and post for post-intervention. These three variables make up the within-subjects factor, time, and the scores within these three variables reflect the dependent variable, cholesterol. The different interventions were stored in the variable, group, where "Control" is the control group, "Int_1" is the low-intensity training intervention, and "Int_2" is the high-intensity training intervention. In variable terms, the researcher wishes to know if there is an interaction between group and time on cholesterol.
In our enhanced mixed ANOVA guide, we show you how to correctly enter data in SPSS Statistics to run a mixed ANOVA. You can learn about our enhanced data setup content here. Alternately, we have a generic, "quick start" guide to show you how to enter data into SPSS Statistics, available here.
The 20 steps below show you how to analyse your data using a mixed ANOVA in SPSS Statistics, including which post hoc test to select to determine where any differences lie, when none of the seven assumptions in the previous section, Assumptions, have been violated. At the end of these 20 steps, we explain what results you will need to interpret from your mixed ANOVA. If you are looking for help to make sure your data meets assumptions #4, #5, #6 and #7, which are required when using a mixed ANOVA and can be tested using SPSS Statistics, we show you how to do this in our enhanced content (see here).
Click Analyze > General Linear Model > Repeated Measures... on the top menu, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
You will be presented with the Repeated Measures Define Factor(s) dialogue box, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
In the Within-Subject Factor Name: box, replace "factor1" with a more meaningful name for your within-subject factor. In this example, replace it with the name "time", as this reflects the within-subject factor, time. Enter into the Number of Levels: box the number of time points (i.e., the number of levels of the within-subject factor). In this case, enter "3", representing pre, mid and post, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
Click the button and you will get the following screen:
Published with written permission from SPSS Statistics, IBM Corporation.
Put an appropriate name into the Measure Name: box. Basically, this is the name of the dependent variable, which is cholesterol in this example. Therefore, enter "Cholesterol" and click the button, and you will end up with the screen below:
Published with written permission from SPSS Statistics, IBM Corporation.
Published with written permission from SPSS Statistics, IBM Corporation.
Transfer pre, mid and post into the Within-Subjects Variables (time): box by highlighting all the variables (clicking on them whilst holding down the shift-key) in the left-hand box and clicking the top button. You will end up with the following screen:
Published with written permission from SPSS Statistics, IBM Corporation.
Transfer the between-subjects factor, group, into the Between-Subjects Factor(s): box, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
Published with written permission from SPSS Statistics, IBM Corporation.
Transfer group from the Factors: box to the Separate Lines: box and time into the Horizontal Axis: box, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
Note: This particular setup works well for this example. However, which factor takes the role of the horizontal axis and which the separate lines for your study is up to you (i.e., whatever makes the most sense to you).
Published with written permission from SPSS Statistics, IBM Corporation.
Published with written permission from SPSS Statistics, IBM Corporation.
Published with written permission from SPSS Statistics, IBM Corporation.
Note: If your between-subjects factor only has two groups, you will not need to run any post hoc tests. Remember, these post hoc tests are for the main effects and not the interaction (i.e., they are not simple main effects).
Published with written permission from SPSS Statistics, IBM Corporation.
Select Studentized from the –Residuals– area, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
Published with written permission from SPSS Statistics, IBM Corporation.
Transfer time and "group*time" (the interaction term) from the Factor(s) and Factor Interactions: box to the Display Means For: box by highlighting them and clicking the button. This will activate the Compare main effects checkbox (i.e., it will no longer be greyed out). Tick this checkbox and select "Bonferroni" from the drop-down menu under Confidence interval adjustment:. Then, in the –Display– area, tick the Descriptive statistics, Estimates of effect size and Homogeneity tests checkboxes. After you have done all this, you will be presented with the following screen:
Published with written permission from SPSS Statistics, IBM Corporation.
The output generated by SPSS Statistics is quite extensive and can provide a lot of information about your analysis. However, if there was a statistically significant interaction between your two factors on the dependent variable, you will need to carry out some additional steps in SPSS Statistics. Below we briefly explain the main steps that you will need to follow to interpret your mixed ANOVA results, and where required, perform additional analysis in SPSS Statistics. If you want to know how to go through all these sections step-by-step, together with the relevant SPSS Statistics output, we show you how to do this in our enhanced mixed ANOVA guide. You can learn more about our enhanced content in general here. First, take a look through these steps:
If you are unsure how to interpret your mixed ANOVA results or how to check for the assumptions of the mixed ANOVA, carry out transformations using SPSS Statistics, or conduct additional SPSS Statistics procedures to run simple main effects on your data (see Step #3a), we show you how to do this in our enhanced mixed ANOVA guide. We also show you how to write up the results from your assumptions tests and mixed ANOVA output if you need to report this in a dissertation/thesis, assignment or research report. We do this using the Harvard and APA styles. You can learn more about our enhanced content here.