Wilcoxon Signed Rank Test vs Paired Student's t-test

The sleep data set

Let’s consider the sleep data set. The data set contrasts the effect of two soporific drugs (aka sleeping pills) by providing the change in the number of hours slept after taking the drug compared to a baseline:

data(sleep) # load the sleep data set
print(sleep)

##    extra group ID
## 1    0.7     1  1
## 2   -1.6     1  2
## 3   -0.2     1  3
## 4   -1.2     1  4
## 5   -0.1     1  5
## 6    3.4     1  6
## 7    3.7     1  7
## 8    0.8     1  8
## 9    0.0     1  9
## 10   2.0     1 10
## 11   1.9     2  1
## 12   0.8     2  2
## 13   1.1     2  3
## 14   0.1     2  4
## 15  -0.1     2  5
## 16   4.4     2  6
## 17   5.5     2  7
## 18   1.6     2  8
## 19   4.6     2  9
## 20   3.4     2 10

extra indicates the increase/decrease (positive/negative values) in sleep compared to the baseline measurement, group denotes the drug, and ID gives the patient ID. To make this clearer, I’ll rename group to drug:

colnames(sleep)[which(colnames(sleep) == "group")] <- "drug"

Note that the sleep data set contains two measurements for every patient. Thus, it is appropriate for showcasing paired tests such as the ones we are dealing with.

What are we testing for?

Assume we are working in a pharmaceutical company and these are the data that were just obtained from a clinical trial. Now, we must decide which of the two drugs you should go forward with for market release. A reasonable way of selecting the drug would be to identify the drug that performs better. More specifically, the question is the following: is one of the drugs associated with greater values of extra than the other drug?

To get an intuition about the effectiveness of the two drugs, let’s plot their corresponding values:

The plot shows that the median increase in sleep time of drug 1 is close to 0, while the median increase for drug 2 is close to 2 hours. So, based on these data it seems that drug 2 is more effective than drug 1. However, we still need to determine whether our finding is statistically significant.

Wilcoxon Signed Rank Test

The Wilcoxon signed rank test uses the sum of the signed ranks as the test statistic \(W\): \[W=\sum _{{i=1}}^{{N}}[\operatorname{sgn}(x_{{2,i}}-x_{{1,i}})\cdot R_{i}]\] Here, the \(i\)-th of \(N\) measurement pairs is indicated by \(x_i = (x_{1,i}, x_{2,i})\) and \(R_{i}\) denotes the rank of the pair. The rank simply represents the position of an observation in an ordered list of \(|x_{2,i} - x_{1,i}|\). The inuition of the test statistic is that pairs with large absolute differences will have large ranks \(R_{i}\). Thus, these pairs are the determining factors of \(W\), while pairs exhibiting small absolute differences have a low \(R_{i}\) and therefore little influence on the outcome of the test. Since the test statistic is based on ranks rather than the measurements themselves, the Wilcoxon signed rank test can be thought of as testing for shifts in median values between two groups.

To perform the test in R, we can use the wilcox.test function. However, we have to explicitly set the paired argument to indicate that we are dealing with matched observations. To specify the one-tailed test, we set the alternative argument to greater. In this way, the alternative of the test is whether drug 2 is associated with larger increases in sleep duration than drug 1.

x <- sleep$extra[sleep$drug == 2]
y <- sleep$extra[sleep$drug == 1]
res <- wilcox.test(x, y, paired = TRUE, 
                   alternative = "greater")

## Warning in wilcox.test.default(x, y, paired = TRUE, alternative = "greater"):
## cannot compute exact p-value with ties

## Warning in wilcox.test.default(x, y, paired = TRUE, alternative = "greater"):
## cannot compute exact p-value with zeroes

x - y

##  [1] 1.2 2.4 1.3 1.3 0.0 1.0 1.8 0.8 4.6 1.4

res$p.value

## [1] 0.004545349

Paired Student’s t-test

The paired Student’s t-test is a parametric test on the means of paired quantitative measurements from two groups. Here, parametric means that the t-test assumes that the mean difference between samples is normally distributed. The test relies on identifying whether the mean difference of measurements from the two groups, \(\bar{X}_{D}\) is larger than \(\mu_D\), where \(\mu_D\) is typically set to 0 in order to find if there is any difference. The test-statistic,

\[\displaystyle t={\frac {{\bar {X}}_{D}-\mu _{0}}{\frac {s_{D}}{\sqrt {n}}}},\]

is normalized using the standard deviation of differences, \(s_D\), and the number of pairs \(n\). By normalizing according to \(\frac{s_D}{\sqrt{n}}\), the test statistic regulates the value of the test statistic according to the number of samples (\(|t|\) increases with more samples) and the standard deviation of differences (\(|t|\) decreases if the deviation increases).

In R, we can perform the paired t-test with the t.test function. Note that t.test assumes that population variances are inequal. In this case, the test is also called Welch’s t-test. To obtain the original t-test, which assumes that the population variances are equal, we can just set the equal.var parameter to TRUE. Here, we will just use with the default setting:

t.result <- t.test(x,y, paired = TRUE, alternative = "greater")
print(t.result$p.value)

## [1] 0.001416445

Again, the p-value is less than 0.05. Thus, we would be inclined to accept the alternative hypothesis: drug 2 is associated with a greater increase in mean sleep duration than drug 1.

diff.df <- data.frame(diff = sleep$extra[sleep$drug == 1] - sleep$extra[sleep$drug == 2])
ggplot(diff.df, aes(x = diff)) + geom_histogram()

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

require(car) # load car package to use 'qqp' rather than native 'qqplot' function

## Loading required package: car

## Loading required package: carData

qqp(diff.df$diff)

## [1] 9 5

Summary: Wilcoxon signed rank test vs paired Student’s t-test

In this analysis, both Wilcoxon signed rank test and paired Student’s t-test led to the rejection of the null hypothesis. In general, however, which test is more appropriate? The answer is, it depends on several criteria:

• Hypothesis: Student’s t-test is a test comparing means, while Wilcoxon’s tests the ordering of the data. For example, if you are analyzing data with many outliers such as individual wealth (where few billionaires can greatly influence the result), Wilcoxon’s test may be more appropriate.
• Interpretation: Although confidence intervals can also be computed for Wilcoxon’s test, it may seem more natural to argue about the confidence interval of the mean in the t-test than the pseudomedian for Wilcoxon’s test.
• Fulfillment of assumptions: The assumptions of Student’s t-test may not be met for small sample sizes. In this case, it is often safer to select a non-parametric test. However, if the assumptions of the t-test are met, it has greater statistical power than Wilcoxon’s test.

Due to the small sample size of the sleep data set, I’d prefer the test from Wilcoxon for these data.

Which test would you use? In general, would you prefer one test over the other?