The idea that prescribing physicians should be guided by the most reliable scientific evidence seems obvious, but the actual methodology of evidence-based medicine was only introduced in the early 1990s by an international group of clinicians and researchers led by Gordon Guyatt. Since then it has provided a new paradigm for the scientific foundation of medicine and has influenced other disciplines outside of medicine, for example, evidence-based psychotherapy, science and government. The novel concept of evidence-based medicine is based on hierarchies of evidence from opinions of respected authorities, mechanistic reasoning (pathophysiologic rationale), and reports of expert committees at the bottom to various levels of observational studies and finally to randomized clinical trials (RCTs) at the apex of the pyramid. Since RCTs provide the most rigorous testing of therapies, they are the gold standard. When treatments long believed to be safe and effective are subjected to RCTs, many turn out to be as useless as the quackery of snake oil or as harmful as mercury. So, attention to RCTs as the evidence informing clinical judgment and practice is perhaps analogous to results of rigorous experiments in physics which turn out to be very different from our intuitions. Everyone, it seems, is on the bandwagon, from medical societies and pharmaceutical companies to general practitioners and surgeons, in the quest for an evidence-based practice.
Since evidence-based medicine was conceived with the practical aim of improving the efficacy of medicine, it does not appear that there is much of philosophical interest in the concept until one raises the epistemological question, as Jeremy Howick does in this book: What is the evidence for evidence-based medicine philosophy of evidence (9)? In other words, what is the evidence that evidence-based medicine is any better in improving patient outcomes than medical practice before the new paradigm? So, evidence-based medicine requires evidence and this is what Howick attempts to achieve in this splendid treatment of the subject. With an eye on the philosophical, Howick is also concerned with various paradoxes that arise concerning the hierarchies of evidence-based medicine; for example, many treatments in whose effectiveness we have the most confidence have never been subjected to RCTs, including simple procedures such as the Heimlich maneuver and tracheotomy. Howick’s thesis is that the evidence-based medicine hierarchies are sustainable provided that we take into account certain modifications, for one that “strict hierarchies should be replaced by the requirement that all evidence of sufficiently high quality should be admitted as evidential support” (xiv, 187). This being the case, we need not subject treatments we know to be effective to RCTs, for this would be as redundant as the need to test parachute effectiveness against parachute placebo.
In order to qualify as good evidence for an evidence-based medicine, outcomes must be clinically effective (clinically significant, rather than merely statistically significant) according to which: (i) patient-relevant benefits outweigh any harms, (ii) the treatment is applicable to the patient being treated, and (iii) it is the best available option (24). The evidence must demonstrate that the patient will live longer or better. Good evidence also rules out plausible rival hypotheses (33). So, RCTs generally maintain their position in the hierarchies because when well-designed and well-conducted (double-masked and randomized), RCTs minimize confounding factors such as the expectation of patients to recover by knowing they are given the experimental treatment. Observational studies cannot meet this standard because they involve observations in routine practice that cannot rule out the confounding factors. Certain observational studies will claim a treatment to be effective and safe but, when subjected to rigorous RCTs, show the very opposite. The same relationship holds between conclusions drawn on the basis of mechanistic reasoning and well-conducted clinical studies. Faced with contradictory conclusions from results in the hierarchies, Howick says it is rational to bet on the results form RCTs since RCTs are less likely to suffer from bias (53).
However, Howick introduces his rule of evidence: comparative clinical studies (or observational studies more generally) provide good evidence when the effect size outweighs the combined effect of plausible confounders (56). In cases in which the effect size is dramatic, potential confounders ruled out by double masking are outweighed by the effect size. So, for example, a retrospective or naturalistic study of the effectiveness of a new antidepressant in which eighty percent of the patients responded to treatment would not be good evidence since confounding factors for subjective ratings could provide alternative explanations, but in a similar test in which antibiotics for meningitis produced dramatic results, potential confounding factors are unlikely to account for the outcome of the study. Generally, RCTs provide better evidence than observational studies but it is not true that RCTs always beat other forms of evidence. Many RCTs will tout statistical significance with a marginal success for the primary outcome measure but fail to demonstrate any clinical significance.
Mechanistic reasoning is inferences about a treatment’s probable efficacy based on an understanding of a cause-effect relationship. In some instances, we have reason to believe that we understand the causal mechanisms; at the very least the success in patient outcomes leads us to believe this is the case. In other instances, the causal mechanisms are unknown or so complex that there are gaps in our knowledge and little confidence in intervention. Where the mechanisms are well understood and there are no gaps in our knowledge, mechanistic reasoning certainly contributes to the total evidence, but by itself, it remains low in the hierarchies. Howick writes: “many lives would have been saved had the requirement for mechanistic reasoning been dropped in cases where we had evidence from high-quality comparative clinical studies” (136). He lists cases in which mechanistic reasoning led to the adoption of therapies that were either useless or harmful. These strongly suggest skepticism about therapeutic claims without support from high-quality evidence from clinical studies (154-56). Howick concludes that high-quality mechanistic reasoning should be weighed alongside results from comparative clinical studies since hypotheses supported by both are less likely to be spurious than hypotheses supported by one type of evidence (135-36).
When Howick turns his attention to expert judgment, it is clear why this form of evidence either remains at the bottom of the hierarchies or is excluded altogether. Reverence for experts lionized by the profession has led to the retention of therapies that are either harmful or useless. Moreover, all clinicians regard themselves as better than average when evidence from studies demonstrates otherwise. Nonetheless, there is a role for expert judgment of skilled clinicians when it comes to knowledge of how to respond to the individual circumstances and values of the patient using the best research evidence (178).
As a bare, abstract structure without regard to the merits and demerits of particular studies, mechanistic reasoning, or expert opinion, evidence-based medicine is therefore inadequate as a guide to clinical practice. One sort of evidence has to be considered within the totality of evidence to determine whether a medical intervention is likely to have clinically relevant benefits.
One of the main obstacles to achieving the ideal of evidence-based medicine is only briefly discussed in the conclusion of Howick’s book, namely, the proverbial fox guarding the henhouse—i.e., the great majority of RCTs are conducted by industry in partnerships with academic physicians. This is perhaps because the problem is deeply practical and political rather than philosophical. Howick draws attention to the fact that industry-sponsored trials are more likely to show positive results than non-industry funded trials (189-191). Due to the intensely competitive profit motive of marketing drugs and medical devices, the industry-sponsored RCTs have become notorious for misrepresentation of the efficacy and safety results. A good example of this was revealed in the manipulation of data to downplay safety results and ‘hide dead bodies’ (statistically speaking) in the clinical trials of Merck’s rofecoxib (Vioxx), which is estimated to have caused up to 120,000 cardiovascular events in the United States, including 40,000 to 60,000 that were fatal.
Prescribing physicians who are well aware of this problem are rightly skeptical about much of what passes for medical knowledge from RCTs, observational studies and respected authorities, i.e., ‘key opinion leaders,’ hired as product champions for the drug companies. Since physicians and researchers are never in position to know which trials are reported honestly and which are infected with statistical spin and blatant misreporting of outcomes, all bets are off regarding the reliability of evidence-based medicine. Where we do get some insight into the extent of the problem is from government inquiries and plaintiff’s lawsuits, but what comes to the surface in the public domain is only the tip of the iceberg. Thousands of medical journal articles reporting the results of industry RCTs cited over and over again remain unchallenged and when the very few subjected to re-analysis are proven to be fraudulent, editors of the journals and their owners refuse retraction.
Howick’s The Philosophy of Evidence-Based Medicine is valuable for the clear and cogent case that he presents for understanding the problems of evidence-based medicine and its solutions. Insofar as his primary goal is a critical evaluation of evidence-based medicine, Howick makes no attempt to explain where in the philosophical spectrum he stands regarding problems of confirmation or falsification of hypotheses or the all-important matter of how the paradigm of evidence-based medicine constitutes an advance, if indeed it does, in the progress of medical science. These issues remain for a more comprehensive philosophical investigation.
University of Edinburgh