Jose de Leon: Training psychiatrists to think like pharmacologists
27. Evidenced-based versus personalized medicine
Jose de Leon’s reply to Donald F. Klein’s commentary
Since the 1980s when I was a psychiatry resident in Spain, I have been an admirer of Dr. Klein’s psychopharmacology articles and textbook. Therefore, I am grateful for his kindness in paying attention to “Lecture 27. Evidence-based versus personalized medicine: Are they enemies?”
As Dr. Klein indicates, this lecture has 169 slides and is based on an editorial (de Leon, 2002). The editorial was published in my favorite psychopharmacology journal: Journal of Clinical Psychopharmacology and has 8,653 words and 150 references. Based on a combination of ideas from four physicians with epidemiological training, Feinstein, Ioannidis, Sacket and Vandenbroucke, I initially wrote a much shorter editorial for psychiatrists pointing out the complexity of a problem, namely, the concept of statistical heterogeneity, which means that outliers are not well represented in the means summarizing randomized clinical trials (RCTs). However, I was criticized by 4 reviewers, including one who was an expert in statistics, so the editor, Dr. Shader, asked for solutions for the complicated problem I was raising. After reading more than 100 articles on statistics and/or epidemiology that I had never read before, I was able to add a second section, centered on solutions, based on combinations of ideas proposed by pharmacologists, epidemiologists, philosophers of science, physicians and by Senn, a statistician, who in my opinion, has the best conceptual understanding of the limitations of randomized clinical trials (RCTs). I was extraordinarily pleased with myself when Vanderbroucke e-mailed me that his first impression of the editorial was that it was “a very scholarly piece of work that deserves closer scrutiny for the new ideas it might bring.” Unfortunately, I soon realized the paradoxical mistake I had made: the editorial was of interest for an epidemiologist like Vanderbroucke but it was too complex for psychiatrists, the original target of my editorial.
Then I developed the PowerPoint lecture and after testing it with several audiences of psychiatrists, I realized that I could not do a good job in summarizing the 8,653-word editorial since it had too many complex messages. I realized that, if I could get psychiatrists to understand the concept of statistical heterogeneity, the original goal of that editorial, I should be extremely happy. This long introduction serves the purpose of explaining why I need to apologize to Dr. Klein who, without reading the editorial, is not likely to understand the PowerPoint completely.
I think that Dr. Klein provides an excellent summary of the core of the presentation that evidence-based medicine (EBM) focuses on average results of RCTs while personalized medicine requires attention to outliers. On the other hand, the next statement, “This outlier emphasis is considered recent and largely attributed to Ioannidis” does not appear correct to me. It appears that my inability to summarize complex concepts has confused Dr. Klein. If he looks at slide 83 it explains that the concept of the “average patient may not represent well all patients”, which is not recent, as it was stressed by Feinstein in many of his books and articles before EBM became a “fad”. However, I would agree completely that what is “recent” is the statisticians’ attempts to provide “a priori” and “a posteriori” solutions (slide 85) and, most importantly, Senn’s contribution providing the best discussion of solutions (slides 86-92).
According to Vickers (2005a), many psychiatrists are highly optimistic but misguided about the use of ANCOVA (analysis of covariance) in RCTs. Vickers (2005a) recommends the use of regression methods, particularly (longitudinal mixed-effects models), rather than ANCOVA. I personally find that longitudinal mixed-effects models have two major advantages over ANCOVA models: 1) they allow calculating effect sizes, and 2) they do not require a normal distribution. It appears to me that Dr. Klein is one of the many psychiatrists who thinks too highly of ANCOVA and believes that it is a good method for addressing statistical heterogeneity in RCTs. The problem with outliers and personalized medicine is that the statistical approach needs to be different depending upon whether the outliers are up to 50%, 10% or 1% of the sample (slide 118). Let’s focus on a sample of 10% to simplify. If we are interested in a sample with 10% outliers this means that we are necessarily interested in patients excluded from a normal distribution. Hopefully the reader remembers that a normal distribution assumes that 95% of the sample is between the mean ±2 standard deviations (SDs). Therefore, if we are focusing on the 10% who are outliers, by definition a normal distribution cannot represent them well. ANOVA (analysis of variance) and ANCOVA techniques were not developed to deal with distributions that are far removed from a normal distribution. As a matter of fact, another of Vickers’s articles (Vickers 2005b) that focuses on simulations demonstrates that in skewed distributions non-parametric methods are better than ANCOVA. Moreover, in extremely skewed distributions, the estimate provided by ANCOVA is of “questionable interpretability”, according to Vickers (2005b).
I am afraid that I completely disagree with the next statement by Dr. Klein, “Further, in the particular case most relevant to Personalized Medicine, when different treatments have non-parallel regression slopes, a priori risk stratification is very difficult. This hurdle is avoided by ANCOVA. However, in neither case are we down to an individual Personalized prescription, rather, for a group, a narrowed predictive set is achieved.” I need to apologize to Dr. Klein that my slides are not sufficiently clear. Slide 137 mentions that my recipe for personalizing RCTs requires using pharmacological mechanisms a priori. That is better explained in the 2012 editorial (de Leon, 2012). Let me describe here a simple version of what I mean by using pharmacological mechanisms a priori. Based on pharmacokinetic knowledge, there is no doubt that a risperidone RCT is going to be contaminated with outliers who are poor metabolizers (PMs) and ultra rapid metabolizers (UMs). These subjects are not represented by the mean dose of the sample since PMs require half of the dosage while UMs require at least double the dosage (Spina and de Leon, 2015). Risperidone PMs can be explained by genetic variation (CYP2D6 PMs) or the co-prescription of powerful CYP2D6 inhibitors (e.g., fluoxetine or paroxetine). Risperidone UMs can be definitively explained by the prescription of potent inducers (e.g., carbamazepine, phenytoin and phenobarbital) and possibly by genetic variations (CYP2D6 UMs). There are two a priori ways of dealing with this. First, the simpler method is to exclude anyone taking potent inducers and/or inhibitors, genotype everyone in the trial and exclude all CYP2D6 UMs and PMs. Thus, the RCT will have excluded all outliers and will represent only subjects with “normal” metabolism and tell us nothing about risperidone dosing in outliers, or personalizing dosing. The second way is to include all subjects or even, if needed, to enrich the samples with outliers and provide stratified dosing. PMs will automatically need to receive half doses and UMs double doses. That is a real personalized medicine approach to risperidone dosing in an RCT. If risperidone UMs and PMs are not taken into account, there is nothing a posteriori that can be done by ANCOVA to correct it. As a matter of fact, an RCT that did not take into account the inductive effects of carbamazepine was unable to establish risperidone as better than placebo for adjunctive treatment of mania in patients taking carbamazepine (Yatham et al., 2003).
I agree 100% with Dr. Klein’s statements: 1) “More important, any individual study does not produce a definitive result. It is the Expectation derived from an unbiased, inclusive, set of well-done studies, that pursue a common goal, that approaches a definitive result.” And 2)“That major, well done studies with negative treatment outcomes have been concealed by profit motivated Pharma, demonstrates that a central methodological assumption has been violated.” Ioannidis’s focus on the problem of false findings is his most important contribution (slide 146).
I agree 100% with Dr. Klein’s first conclusion, “Our confidence in using Evidence Based Medicine for therapeutic guidance is misplaced.” If any reader wants to read more about it, I recommend one of the most recent commentaries by Ioannidis (2016).
I agree 100% with Dr. Klein’s second conclusion, “Our hopes for Personalized Medicine are subverted.” Two of my editorials explain how: 1) pharmaceutical companies have subverted personalized medicine (de Leon, 2012), and 2) pharmacogenetic companies marketing non-validated tests are now further complicating the area of personalized medicine (de Leon, 2016).
Regarding the last paragraph, “Regulatory systems, like the FDA and EMA, attempt to keep our assumptions reasonably correct so eventual conclusions are credible. Meeting the goal of public, unbiased, independent, expert review of all relevant trials has failed. Therefore, extension of regulatory powers is necessary. That conclusion should be addressed in any currently informed discussions of Evidence Based or Personalized Medicine. However, it is absent from this paper, as well as in many parallel, apparently sophisticated, discussions.” As far as I can tell, Dr. Klein has not read any of my four editorials that criticize: 1) the lack of interest by the FDA in improving package inserts/prescribing information (de Leon, 2011); 2) the difficulties of regulatory agencies in regulating RCTs (de Leon, 2012); 3) the corruption of pharmaceutical companies and psychiatric academia, which has negatively impacted psychiatric practice (de Leon, 2014); and 4) the lack of action of regulatory agencies regarding commercial pharmacogenetic tests (de Leon, 2016).
I want to conclude by thanking Dr. Klein, one of my heroes in psychopharmacology during my youth, who has been kind enough to spend his time reviewing my presentation. I also must apologize to the reader for the need to present complex statistical concepts usually reserved for statistics journals (such as non-normal distributions, non-parametric tests, and longitudinal mixed-effects models), to answer Dr. Klein’s comments.
References
de Leon J. Highlights of drug package inserts and the website DailyMed: the need for further improvement in package inserts to help busy prescribers. Journal of Clinical Psychopharmacology 2011; 31: 263-265.
de Leon J. Evidence-based medicine versus personalized medicine: are they enemies? Journal of
Clinical Psychopharmacology 2012, 32: 153-164. Pre-published free version:
http://uknowledge.uky.edu/psychiatry_facpub/41/
de Leon J. Paradoxes of US psychopharmacology practice in 2013: undertreatment of severe mental illness and overtreatment of minor psychiatric problems. Journal of Clinical Psychopharmacology 2014; 34: 545-548.http://uknowledge.uky.edu/psychiatry_facpub/25/
de Leon J. Pharmacogenetic tests in psychiatry: from fear to failure to hype. Journal of Clinical Psychopharmacology 2016; 36: 299-304.
Ioannidis JP. Evidence-based medicine has been hijacked: a report to David Sackett. Journal of Clinical Epidemiology 2016; 73: 82-86.
Spina E, de Leon J. Clinical applications of CYP genotyping in psychiatry. Journal of Neural Transmission 2015; 122: 5-28.
Yatham LN, Grossman F, Augustyns I, Vieta E, Ravindran A. Mood stabilisers plus risperidone or placebo in the treatment of acute mania. International, double-blind, randomised controlled trial. British Journal of Psychiatry 2003; 182: 141-147.
Vickers AJ. Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosomatic Medicine 2005a; 67: 652-655.
Vickers AJ. Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Medical Research Methodology 2005b; 5: 35.
Jose de Leon,
October 6, 2016