The structure of surveys and the peril of panels.

Abstract

University league tables give the image that there is a single dimension along which institutions can be placed. Most derive rankings from an aggregate score of multiple items, which often include opinion responses which has the potential to introduce sampling bias. This paper explores what happens when the two issues of dimensionality and sampling bias interact in league tables. It uses the Times Higher Education Student Experience Survey – which uses a panel design – to explore these issues. It notes that they combine to produce a distorted image of the relative quality of the student experience in different institutions. We conclude that ignoring dimensionality and the systematically unrepresentative nature of the sample could lead policy makers to draw inappropriate conclusions.