Down with the Hierarchies

An evidence hierarchy is a rank ordering of kinds of research methods according to the potential for those methods to suffer from systematic bias. Evidence hierarchies are widely used to assess evidence in systematic reviews of medical studies. Chapter 5 gives several arguments against the use of evidence hierarchies. The problems with evidence hierarchies are numerous, and include methodological shortcomings, philosophical problems, and formal constraints. Medical science should not employ evidence hierarchies, including even the latest and most sophisticated of such hierarchies. There is an advantage to evidence hierarchies, namely, their apparent rigidity, which could constrain researcher degree of freedom when performing a review of available evidence. But this constraint comes at the cost of an overly simple conception of evidence, and does not in fact constrain researcher degree of freedom in a substantive way.