Martin M. Katz: Depression and Drugs Neurobehavioral Structure of a Psychological Storm
Donald F. Klein’s final comment
The requested 2X2 data layout, as presented in "Martin M. Katz’s response to Donald F. Klein’s reply to Carlos Morra’s comment" (INHN Controversies 10.15.2015) presented in a parallel project (Martin M. Katz: Onset of antidepressant action) were insufficiently identified, as Leslie Morey agreed (Controversies 12.12.2015). The ambiguity is the uncertainty about data in which row of the table should be considered as early improvement.
Assuming early improvement refers to row 2, this table roughly agrees with Marty's statement that "70% of patients showing early improvement would go on to respond at 6 or 8 weeks".
Hamilton Rating Scale
Late Response
<50% >50%
Early Response <20% 15 2
>20% 8 25
Note, 33 are predicted to do well but only 27 (82%) actually did do well. Based on Marty's within drug analysis the drug is overvalued. That this is considered clinically significant, is arbitrary. Such a clinical judgment should be stipulated prior to the investigation.
One might be interested in the possibility that a very low pre-score would indicate a likely treatment shift. However, even better, such a score should allow a drug-free period of clinical watchful waiting.
The, hopefully predictive, correlation (0.6) between pre- and post-measures, accounting for 36% of the variance, is generally considered too low for predictive use.
Further problems remain. The "active drug" sample, N = 50, combines the Paroxetine study (N=24) with the DMI study (N=26). No justification is given. The combination of Paroxetine, picked as a serotonergic agent and DMI as a noradrenergic agent, requires a priori justification. Apparently an increase in sample size was considered necessary.
Marty provided placebo data used by Morey. This allows progress from a predictive study, derived entirely from within drug data, to an estimate derived from contrasting drug vs placebo.
Drug Placebo
Recover 27 6
Not Rec 23 13
Chi-square = 2.77 Trend p=0.09, 2-Tailed
In any case, an analysis focused on invalidating the null hypothesis does not answer the question with sufficient strength to be a useful predictor. The correlation, 0.6 found here has 95% confidence limits of 0.39, 0.72.
So the upper limit of the correlation remains insufficient for predictive utility, even if one stacks the dice by an untrue assumption of sample bivariate normality.
Katz's argument is destroyed by the insignificant contrast between drug and placebo outcomes. Even strong findings, if derived from a small data set, would call for large sample replication before allowing interpretation as sound predictions about the useful length of definitive clinical trials.
That this insignificant, 6-week, drug vs. placebo contrast justifies the utility of a much shorter clinical trial is preposterous. Katz's claim that larger studies have already agreed with his conclusions needs more than an article reference. The exact analyses allowing parallel conclusions must be pointed out. I have failed to find them.
It is also illogical for large supposedly definitive trials to be followed by a small trial, that at best could add nothing new.
Donald F. Klein
July 28, 2016