The "paired-samples sign test", typically referred to as just the "sign test", is used to determine whether there is a median difference between paired or matched observations. The test can be considered as an alternative to the dependent t-test (also called the paired-samples t-test) or Wilcoxon signed-rank test when the distribution of differences between paired observations is neither normal nor symmetrical, respectively. Most commonly, participants are tested at two time points or under two different conditions on the same continuous dependent variable. However, two different groups of participants are possible as part of a "matched-pairs" study design.
For example, you could use the sign test to understand whether there was a median difference in smokers' daily cigarette consumption before and after a 6-week hypnotherapy programme (i.e., your dependent variable would be "daily cigarette consumption", with the two time points being "before" and "after" the hypnotherapy programme). You could also use the sign test to determine whether there was a median difference in reaction times under two different lighting conditions (i.e., your dependent variable would be "reaction time", measured in milliseconds, and the two conditions would be testing reaction time in a room using "blue light" and a room using "red light").
This "quick start" guide shows you how to carry out a sign test using SPSS Statistics, as well as interpret and report the results from this test. However, before we introduce you to this procedure, you need to understand the different assumptions that your data must meet in order for a sign test to give you a valid result. We discuss these assumptions next.
When you choose to analyse your data using a sign test, part of the process involves checking to make sure that the data you want to analyse can actually be analysed using a sign test. You need to do this because it is only appropriate to use a sign test if your data "passes" four assumptions that are required for a sign test to give you a valid result. You cannot test these assumptions with SPSS Statistics because they relate to your study design and choice of variables. However, you should check whether your study meets these four assumptions before moving on. If these assumptions are not met, there is likely to be a different statistical test that you can use instead. These four assumptions are explained below:
If your study design and data meets these four assumptions, you can run the SPSS Statistics procedure for the sign test, which we illustrate in the Test Procedure in SPSS Statistics section. First, we set out the example we use to explain the sign test procedure in SPSS Statistics.
A researcher wants to test a new formula for a sports drink that improves running performance. Instead of a regular, carbohydrate-only drink, this new sports drink contains a new carbohydrate-protein mixture. The researcher would like to know whether this new carbohydrate-protein drink leads to a difference in performance compared to the carbohydrate-only sports drink. To do this, the researcher recruited 20 participants who each performed two trials in which they had to run as far as possible in two hours on a treadmill. In one of the trials they drank the carbohydrate-only drink and in the other trial they drank the carbohydrate-protein drink. The order of the trials was counterbalanced and the distance they ran in both trials was recorded.
Therefore, for a sign test, you will have two variables. In this example, these are: (1) carb, which is the distance run (in km) in two hours for the carbohydrate-only trial; and (2) carb_protein, which is the distance run (in km) in two hours for the carbohydrate-protein trial. The researcher would like to determine whether there was a difference in the distance run between the two trials, and therefore, if there is a performance difference between the two different sports drinks. In variable terms, the researcher wants to know if the median of the differences between the carb and carb_protein scores is 0 (zero).
In our enhanced sign test guide, we show you how to correctly enter data in SPSS Statistics to run a sign test. You can learn about our enhanced data setup content here or access the enhanced sign test guide by subscribing to the site here. In the next section, we take you through the sign test procedure using SPSS Statistics.
The four steps below show you how to analyse your data using a sign test in SPSS Statistics. At the end of these four steps, we show you how to interpret the results from this test.
Click Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples... on the main menu:
Note: We show you the legacy procedure in SPSS Statistics to run the sign test below since this can be used with older versions of SPSS Statistics (17 and below), as well as more recent versions (version 18 and above). However, you can also run the sign test using the new procedure in SPSS Statistics, which is available for versions 18 and above. This newer procedure provides additional statistics and more graphical options than the legacy procedure. We show you how to run the new procedure and interpret and report the output from it in our enhanced sign test guide. You can access the enhanced sign test guide by subscribing to the site here.
You will be presented with the Two-Related-Samples Tests dialogue box, as shown below:
You need to transfer the variables carb and carb_protein into the Test Pairs: box by highlighting both variables (clicking on both whilst holding down the shift-key) and clicking the button (N.B., you can also transfer each variable separately). You will end up with a screen similar to the one below:
Explanation: The above instructs SPSS Statistics to calculate the Wilcoxon signed-rank test on carb_protein minus carb. As such, we need to change this default, which we do in the next step.
Deselect Wilcoxon and select Sign in the –Test Type– area. You will end up with the following screen:
Click the button to generate the output.
SPSS Statistics will generate quite a few tables of output for a sign test. In this section, we show you the three main tables required to understand your results from the sign test procedure, assuming that no assumptions have been violated. You should start by interpreting median values and paired differences.
Before we delve into the statistical results of the sign test, it is best if we first get our bearings by examining the median values we have generated. There are median values for the two trials (i.e., the two groups of the independent variable) and for the paired differences. This information is presented in the Report table, as shown below (the procedure to produce this table is included in our enhanced sign test guide):
Published with written permission from SPSS Statistics, IBM Corporation.
You can see that the table contains the medians of the variables carb and carb_protein, as well as the median of the differences (difference) (i.e., the column names reflect the names of the variables).
You should evaluate the number of positive, negative and tied paired differences to understand each participant's (relative) response to the two trials. These paired differences will also give you an indication of what to expect for the result of the sign test (remembering that the test is based on the signed differences). This information is provided in the Frequencies table, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
You can see how many participants decreased (the "Negative Differences" row), improved (the "Positive Differences" row) or witnessed no change (the "Ties" row) in their performance in the carbohydrate-protein trial (i.e., carb_protein) compared to the carbohydrate-only trial (i.e., carb).
We can now move on to discovering whether the median of the difference in distance ran between the trials is statistically significant using these signed differences. The result of the sign test is found in the Test Statistics table, as shown below:
Published with written permission from SPSS Statistics, IBM Corporation.
The statistical significance (i.e., p-value) of the sign test is found in the "Exact Sig. (2-tailed)" row of the table above. However, if you had more than a total of 25 positive and negative differences, an "Asymp. Sig. (2-sided test)" row will be displayed instead. We explain the differences between the two ways that the p-value is calculated in our enhanced sign test guide.
Based on the results above, we could report the results of the study as follows:
Twenty participants were recruited to understand the performance benefits of a carbohydrate-protein versus carbohydrate-only drink on running performance as measured by the distance run in two hours on a treadmill. An exact sign test was used to compare the differences in distance run in the two trials. The carbohydrate-protein drink elicited a statistically significant median increase in distance run (0.113 km) compared to the carbohydrate-only drink, p = .004.
In our enhanced sign test guide, we: (a) show you how to interpret and write up the results of the sign test irrespective of whether you ran the legacy procedure (as illustrated in this guide) or the newer procedure in SPSS Statistics; (b) provide a more detailed explanation of how to interpret median values and paired differences, as well as positives, negatives and ties, and finally, exact and asymptotic p-values; and (c) illustrate how to write up the results from your sign test procedure if you need to report this in a dissertation/thesis, assignment or research report. We do this using the Harvard and APA styles. You can access our enhanced sign test guide, as well as all of our SPSS Statistics content, by subscribing to the site here, or learn more about our enhanced content in general here.