Transitivity, scale-invariance, and rank tests

Abstract:

Nearly all rank tests turn out to have the 'rock-paper-scissors' property: they need not be transitive.  I discovered this while trying to work out what the Wilcoxon rank-sum test really meant, in order to explain it in an introductory statistics course.  I will explain why non-transitivity happens, and why it is just an extreme case of the more general problem that rank tests are invariant to monotone transformations.  The fact that rank tests compare distributions without relying on interval-level or ratio-level scores is often regarded as a feature.  I will argue that it is a bug.

 Most of this talk will assume no more than simple undergraduate statistics, although there will be cameo appearances by order topologies, separability, and Arrow's impossibility theorem.