This is a reply to this comment

Sample size is always an issue in clinical research. The target sample in clinical studies areusually patients suffering from one of a range of mental disorders. When investigating a causal or structural factor in the makeup of the disorder or the effect of a treatment, the investigator strives to assemble a representative sample of the disorder, not easy to accomplish. Whatever the study results, however, they must be limited in their generality to the kinds of patients represented in the study.  Second to representativeness of the sample, in accord with the study aims, is the consideration of sample size.  Certain technologies to be applied to analyzing the data require a minimum number of subjects so that not achieving a required size does not allow the statistical techniques appropriate to the problem to be applied. Factor analysis or principal components analyze the relationships among multiple variables. Depending on the precision with which these variables are measured, and the sheer number of variables at issue, factor analytic procedures require rather large samples to produce stable solutions. So clinical research more so than basic research is burdened because of the complexity of its human subjects,the need to assemble large, diversified samplesand to usually follow them over extended lengths of time. In evaluating the factors (dimensions) the viewer must take into account thecontent and quality of the methods utilized to derive them, and note, that in the end, their value is dependent on how well they meet the aims of the overall study.

The viewer will note that in the NIMH Collaborative Depression Study (CDS) (Maas et al 1980) the factors made possible the testing of neurobehavioral hypotheses and, refined analyses of the drug actions upon the disorder.  Their application resulted in new information about the composition of the disorder, about the timing and specificity of clinical actions of the drug, and of their associations with the underlying neurochemical changes affected by these drugs.

The problem initially confronting investigators in that study, based on the aims in the CDS of testing neurobehavioral hypotheses and the effects of treatment, referred to by Klein.was to assemble a "representative" sample, diverse enough to cover the variations across the most severe of depressed patients. If such a group could be assembled and sound, psychometrically tested methods applied to the analysis of their psychopathology, it should be possible to uncover the essential mood, behavior, and cognitive components that comprise the disorder.  And then through principal components analysis identify the underlying dimensions that describe this structure.

How large and diverse a sample must be assembled to meet these aims? We note as background, that because of the practical difficulties in this field noted, clinical studies usually progress on the shoulders of very small samples. So theoretical ideas, like the “catechol amine hypothesis" or the “dexamethasone test”, were developed from relatively small samples. The CDS sample in this area of research was designed to be especially large and diverse in order to generate more definitive tests of these hypotheses, originally developed on small samples.

Six hospitals in diverse areas of the country were recruited and representative samples of unipolar and bipolar depressives, utilizing the research diagnostic criteria (RDC), operational definitions of the disorders, for selection,resulted in 130 patients, a “very large” sample in this sphere of research for this study.

It was possible to use 73 of these patients for the second-order factor analysis of the behavioral components. The sample size requirements for factor analysis are based as noted on the number of variables, the soundness of the methods, so that 5 to 10 patients per variable is required for “exploratory" or confirmatory factor analyses (Floyd & Widaman 1995). The sample size used in the CDS study is not large for factor analysis (conducted with 11 variables) but adequate in accord with technical requirements. Probably more telling is that the variables included are not simply items, known to have dubious reliability, but are previously validated clusters of item score sub-factors already tested for reliability. The methods were selected based on prior factor and other analyses involving proposed dimensions of the disorder uncovered in earlier research, and room was left in the analysis for the derivation of new 2^nd order dimensions to appear in the new sample.

The principal components analysis is the most used, most precise technique available for such analyses. In evaluating the factors (dimensions) the viewer must take into account the quality of the methods used and note that ion the end, the methods’ validities are dependent on prior psychometric analyses, and then on how well they do in meeting the aims of the overall study.

The viewer will note that the factors make possible the testing of the hypotheses, the refined analyses of drug actions on the disorder, resulting in new information about composition of the disorder, about the timing and specificity of clinical actions of the drugs, and their associations with the immediate neural changes effected by the drugs.

Of most importance, however, is that the analyses have made”visible” a conflict of opposed emotional dimensions in this disorder which provides the basis for a new theory of its neurobehavioral dynamics. I expect, in the future, further elaborations on these dimensions and understanding of the “psychological storm” underlying the tumult and severity associated withthis range of disorders.

Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment 1995; 7: 286 - 99.

Maas JW, Koslow SH, Davis JM, Katz MM, Mendels J, Robins E, Stokes P, Bowden C. Biological component of the NIMH Collaborative Program on the psychobiology of depression: Background and theoretical considerations. Psychological Medicine 1980; 10, 759-76.

Martin M. Katz

March 6, 2014