The Kruskal-Wallis test: a complete guide

Kruskal-Wallis pools every observation from all groups, ranks them from lowest to highest, and asks whether the ranks are distributed evenly across the groups. If one group's observations cluster at the high end of the ranking, its rank sum will be larger than chance alone would produce.

The test statistic is H, and it is approximately chi-square distributed with k − 1 degrees of freedom, where k is the number of groups. Like chi-square and F, it cannot be negative and only large values reject the null.

The word “non-parametric” causes some confusion here, because it is often heard as “assumption-free”. It is not. What it means is that the test does not assume the data follow a particular named distribution — there is no normality requirement, no parameter such as a mean or standard deviation being estimated. Independence is still required absolutely, and the shape condition still governs what your conclusion may claim. Rank-based tests trade distributional assumptions for a small loss of power and a slightly weaker conclusion, not for freedom from assumptions altogether.

Note the sixth row. If your problem is unequal variances rather than non-normality, Welch's ANOVA is usually the better answer than switching to ranks, because it keeps the interpretation in the original units and costs less power. Kruskal-Wallis is the tool for the measurement level and the shape of the distribution, not a general-purpose fix for anything that fails a diagnostic test.

The final row is the same caveat that applies to Mann-Whitney, and it is skipped just as often. Strictly, Kruskal-Wallis tests whether the groups differ in the distribution of ranks — whether observations from one group tend to exceed those from another. It reduces to a statement about medians only when the distributions share a similar shape and spread.

Note also what is not assumed: normality is not required, and neither is equality of variance in the sense ANOVA requires it. What matters is that observations are independent and rankable.

Fifteen participants are split across three conditions and rated on a task measure. Pooled and ranked across all 15, the ranks fall as follows.

Step 1 — check the rank total

The ranks must sum to N(N+1)/2 = 15 × 16 / 2 = 120. Here 22 + 42 + 56 = 120.

Step 2 — apply the formula

H = [12 / (N(N+1))] × Σ(Rₖ²/nₖ) − 3(N+1)

= [12 / (15 × 16)] × 1076.8 − (3 × 16)

= 0.05 × 1076.8 − 48 = 53.84 − 48 = 5.84

Step 3 — the decision

df = k − 1 = 2. The critical chi-square value at α = .05 with 2 degrees of freedom is 5.99. Our H = 5.84 falls just short, giving p = .054.

What the near-miss would need to reach significance

It is worth seeing how sensitive this is to sample size. The pattern of ranks above is close to a perfect separation of the three groups, and it produces H = 5.84 against a critical value of 5.99. Had the same ordering held with seven participants per group rather than five, H would rise well past the threshold, because H scales with N while the critical value depends only on the number of groups. Nothing about the effect would have changed — only the evidence available to detect it.

This is the practical argument for running a power analysis before collecting data rather than explaining a null result afterwards. A study that stops at five per group is very likely to produce exactly this outcome: an orderly pattern, a large effect size, and a p-value just the wrong side of the threshold.

Ties

Where observations tie, each receives the average of the ranks they would have occupied, and H must then be divided by a tie correction factor. Ties inflate the correction and therefore increase H slightly. Software applies this automatically; the hand formula above does not, so with substantial tying — common in Likert data — the hand result will be marginally conservative.

A significant H means at least one group differs from at least one other, and nothing more specific. As with ANOVA, stopping there leaves the analysis incomplete.

A point that catches people out: it is possible for the omnibus test to be significant while no pairwise comparison survives correction, and for the reverse to happen too. The first case usually means the differences are spread thinly across several groups rather than concentrated in one pair. The second means the omnibus test lacked power relative to a specific focused comparison. Neither is an error — they are different questions — but both need reporting honestly rather than quietly presenting whichever result is more convenient.

Dunn's test is preferred because it uses the ranks from the original pooled ranking rather than re-ranking each pair, which keeps it consistent with the omnibus test. Running separate Mann-Whitney tests re-ranks within each pair, so the comparisons no longer align exactly with the test that licensed them.

Kruskal-Wallis sits in a family of rank-based tests, each matched to a specific design. Picking the wrong member is common, and the error is usually about independence rather than about measurement level.

There is also a question of what to do when groups are very unequal in size, which arises often with naturally occurring groups such as departments or wards. Kruskal-Wallis handles unequal group sizes without difficulty — the formula divides each squared rank sum by its own group size, which is exactly the adjustment needed. What it cannot do is compensate for a group so small that its rank sum carries almost no information. With one group of 40 and another of 4, the test remains valid but the comparison involving the small group is effectively uninformative, and it is worth saying so rather than reporting all pairwise results as though they were equally well established.

Why Friedman is not interchangeable

If the same participants are measured under all three conditions, their scores are correlated: a person who scores highly in one condition tends to score highly in the others. Kruskal-Wallis assumes every observation is independent, so applying it to repeated measures treats 15 observations from 5 people as though they came from 15 different people. The result is a test with the wrong degrees of freedom, and one that discards the pairing which was the design's main source of power.

Friedman handles this by ranking within each participant rather than across the whole sample. Each person's three scores are ranked 1 to 3, and the test asks whether particular conditions consistently attract the higher ranks. That within-subject ranking is what preserves the pairing.

Following up a Friedman test

The same logic applies as for Kruskal-Wallis: a significant result identifies that some condition differs without saying which. Follow it with pairwise Wilcoxon signed-rank tests, corrected for the number of comparisons, or with Conover's post-hoc procedure.

A p-value says nothing about magnitude, and two effect sizes are commonly reported for Kruskal-Wallis.

For our example: ε² = 5.84 / ((225 − 1)/16) = 5.84 / 14 = 0.417, and η² = (5.84 − 2) / 12 = 0.320. Both suggest a large effect — which sits oddly beside a non-significant p-value, and illustrates the point that with five per group the study lacked the power to establish an effect it may well have detected.

Note that both formulas can return a small negative value when H is very small — smaller than would be expected by chance alone. Report such a result as zero rather than as a negative effect size, since a negative proportion of variance explained has no meaning.

One further caution about interpreting a large effect size alongside a non-significant test, as in the worked example above. It is tempting to describe such a result as “approaching significance” or as a trend. Neither phrase is defensible: the test either rejected the null at your stated threshold or it did not. What you can legitimately say is that the point estimate suggests a substantial difference which this sample was too small to establish, and that the study was underpowered. That framing is accurate, useful to whoever designs the follow-up, and does not smuggle in a conclusion the data cannot support.

1. Claiming a difference in medians without checking shape

Valid only when the distributions have similar shapes. Otherwise the claim is that one group tends to score higher.

2. Stopping at a significant H

Run Dunn's test. Without it you cannot say which groups differ, and the analysis is unfinished.

3. Running pairwise Mann-Whitney tests without correction

Three comparisons at α = .05 gives about a 14% chance of a false positive.

4. Using it on repeated measures

Kruskal-Wallis needs independent groups. For the same participants measured three or more times, use the Friedman test.

5. Reporting means

Report medians and interquartile ranges, consistent with the test.

6. Choosing it as a general fix for failed assumptions

If the problem is unequal variances rather than non-normality, Welch's ANOVA usually serves better.