David Healy: Do randomized clinical trials add or subtract from clinical knowledge
Ken Gillman: A follow-up to the exchange between David Healy and Charles Beasley
P-hunting: The illogical in pursuit of the indefensible*
*Apologies to Oscar Wilde [foxhunting: the unspeakable in pursuit of the uneatable]
This discussion has become a peregrination of protracted prolixity — I do not suppose there will be (m)any with the fortitude to digest Beasley’s 26,000 words. Therefore, I am adding, as briefly as possible, to my previous concise comment (Gillman 2021). The above alliteration of “Ps” is a satirical reflection on the excessive, essentially unscientific, reliance on P-values that pervades RCTs and has been railed against repeatedly by statisticians (Greenland, Senn, Rothman et al. 2016).
I agree with Professor Healy and have made similar points in my own writings.
I repeat this simple fact about the definitive meta-analysis of “quality” RCTs concerning antidepressants, the so-called “Lancet 21 anti-depressant” study (Cipriani, Furukawa and Salanti 2018), which impels only one conclusion: that this enormous, mediocre quality, predominantly industry funded, RCT-work provides negligible useful information or guidance to doctors — and no useful information whatsoever about the real efficacy, mode of action or pharmacology of the drugs concerned: that is what real science must be about; not interminable comparisons of colored jellybeans on the scores produced by some mediocre rating scale.
However, Beasley and his colleagues have served to get many marginally useful “me-too” drugs over the finishing line for FDA approval — to the benefit of the patients? or mainly to the employees, stockholders and KOLs of pharmaceutical companies?
Such meagre progress after 50 years and it takes Beasley 26,000 words to suggest what, yet still, needs to be done to improve it. If he and his associates could not “get it right” after 50 years and devoting much of their careers to the enterprise, that suggests it is an unhelpful endeavor. Is there a sound basis justifying a continuation of their efforts? What chance of progress with RCTs? The 22nd century?
Beasley parades a preoccupation, obsession even, with what Hill did or did not say or mean, nearly one century ago: have things not moved on a little since then? For instance, with the work of the Turing prize-winner Judea Pearl, that I mentioned in my previous comment and which Beasley does not mention, or with the improving appreciation of Bayesian reasoning.
Incidentally, I do seem to remember that Hill’s report on illness in the cotton mills contained no statistics whatsoever; he said that double-blind trials and randomization were only necessary if the differences were small.
Small, miniscule, non-existent: that covers pharmaceutical company RCTs of psychotropics. Hill also made a point of endorsing Claude Bernard’s view that there is “no qualitative epistemic difference between experiment (RCTs) and observation (clinical science/experience).” How presumptuous to implicitly demean the value of that clinical experience and experimentation with the dubious claim of “gold-standard” status for RCTs. RCTs are akin to fiddling with the fine-tuning knob when the radio-receiver is on the wrong waveband.
Pearl said, “Science is nothing without causality” and RCTs contribute nothing to understanding causality — they are impotent in that regard.
Buried in Beasley’s 26,000 words of text is the statement: “… RCT is the most robust method (gold-standard) for advancing sound clinical knowledge.” I profoundly disagree with that, as do many scientists. Here are just a few significant supporting references and comments: “autonomous of the basic sciences… blind to mechanisms of explanation and causation,” Ashcroft (2004); “Emphasis on EBM has eclipsed other necessary research methods in medicine,” Solomon (2011); “we have overshot the mark with EBM,” Berwick (2005); “the notion that evidence can be reliably placed in hierarchies (as all guidelines do) is illusory,” Rawlins (2008).
I contend that the meagre advances RCTs have contributed to clinical science/knowledge are an eloquent testimony to the validity of these assertions. The advances of substance in clinical practice and management over the history of psychopharmacology have come from clinical science and clinical observation — I cannot think of any from RCTs. Contrariwise, RCTs have undoubtedly led to much serious misinformation and misdirection that has done a great deal of harm to patient care and management. Those not involved in patient care and management may not have such a keen appreciation of that life and death issue.
Finally, Beasley curiously says, “after I read past [sic] what I consider Dr. [sic]* Healy’s rather vitriolic descriptions of his concerns with RCTs…”
*I insert “sic” here solely to note Beasley’s repeated use of “sic” to draw gratuitous attention to what he considers are other’s (spelling) errors, by which he reveals both his own pedantry and his knowledge of the history of English and spelling. Since Healy is a professor, could he not have afforded him the courtesy of using that title?
I cannot be alone in failing to understand how he can characterize Healy’s comments as vitriolic (OED “acrimonious, caustic, scathing”) unless he has a latent “sensitivity” of conscience concerning his association with the pharmaceutical industry — after all, he fails to mention the consistent and persistent lack of honesty and probity of an embarrassing proportion of the people with whom he, presumably, “rubbed shoulders” in his work. I could make justifiably vitriolic comments about how widespread and persistent dishonest behavior has wasted a great deal of my valuable time and energy in sifting through these deceitful and fraudulent publications in search of quality data, objectivity and truth, e.g., Gillman 2005, 2006.
References:
Ashcroft RE. Current epistemological problems in evidence based medicine. J Med Ethics 2004;30(2):131-5.
Berwick DM. Broadening the view of evidence-based medicine. Qual Saf Health Care 2005;14(5):315-6.
Cipriani A, Furukawa TA, Salanti G. Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. Lancet 2018;391(10128):1357–66.
Gillman K. Comment. David Healy: Do randomized clinical trials add or subtract from clinical knowledge. inhn.org.controversies. June 3, 2021.
Gillman PK. Drug interactions and fluoxetine: a commentary from a clinician’s perspective. Ex Op Drug Saf 2005;4(6):965-8.
Gillman PK. A systematic review of the serotonergic effects of mirtazapine: implications for its dual action status. Human Psychopharmacology. Clinical and Experimental 2006;21(2):117-25.
Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 2016;31(4):337-50.
Rawlins M. De testimonio: on the evidence for decisions about the use of therapeutic interventions. Lancet 2008;372(9656):2152-61.
Solomon M. Just a paradigm: evidence-based medicine in epistemological context. European Journal for Philosophy of Science 2011;1(3):451.
September 30, 2021