Charles M. Beasley, Jr and Roy Tamura: What We Know and Do Not Know by Conventional Statistical Standards About Whether a Drug Does or Does Not Cause a Specific Side Effect (Adverse Drug Reaction)


6.    Incidences of AEs of real-world interest and limitations on “proof” of presence or absence of an ADR


           How relevant to clinical reality and what would be relevant to both prescribers and patients who might suffer a major (i.e., life-threatening or fatal) ADR is our hypothetical example of an ADR that occurs with an incidence of 1 in 1,000 persons treated but virtually never happens in an untreated population.  Aplastic anemia and the spectrum of Stevens-Johnson Syndrome (SJS) - toxic epidermal necrolysis (TEN) afford a context for considering the relevance of our example.

           A major, international study of both agranulocytosis and aplastic anemia has been conducted under the sponsorship of the WHO.  The first report described rates of occurrence for aplastic anemia ranging across seven sites from 0.6 to 3.1 (adjusted mean: 2.2) per million-person-years (International Agranulocytosis and Aplastic Anemia Study 1987).  A more recent report of this study reported a range of rates of cases from 0.7 to 4.1 per million-person-years (Kaufman, Kelly, Issaragrisil et al. 2006).  For aplastic anemia, about 25-40% of cases are considered due to exogenous exposures (drugs, toxic substances) or other external factors and the majority are believed to be idiopathic and have no identifiable etiology (Kaufman, Kelly, Issaragrisil et al. 2006).  Therefore, with aplastic anemia, the incidence on an annual basis (~2-3 / million) is much lower than the 1 in 1,000 in our example.  Furthermore, some background incidence of aplastic anemia would be expected due to idiopathic factors and exposures to substances other than test drug and thus would further increase sample sizes required to “prove” causation by a drug. 

           Stevens-Johnson Syndrome and toxic epidermal necrolysis are the extreme manifestation of the continuum of the clinical diagnoses of erythema multiforme (EM) – SJS – TEN; all share some characteristic histopathological feature of epidermal necrolysis (there is some disagreement in grouping erythema multiforme as a separate clinical entity or as part of the spectrum).  A large UK epidemiological study reported the rate for combined SJS-TEN as 5.6 (95% CI: 5.31-6.30) per million-person-years (Frey, Jossi, Bodmer et al. 2017)).  A separate, large national epidemiological study in South Korea reported rates for SJS of 3.96-5.03 per million person-years (range for individual years across four years) and rates for TEN ranging from 0.94-1.45 per million person-years (Kang, Ko, Kim et al. 2015).  The UK and Korean results are comparable for rates of combined SJS and TEN.  The incidence of SJS – TEN is then in the range of ~6.5 per million-person years.  In contrast to aplastic anemia, because SJS – TEN is primarily due to exogenous exposure, background rates could approach 0 if a study could be conducted where study subjects receiving the active investigational drug received no other medications and control subjects receiving placebo received no drugs.  Of course, this would be a highly impractical study design, especially given the enormous number of subjects required for definitive assessment

           While “proving” by conventional statistical standards that a test drug does or does not cause a specific ADR with an incidence of 1 in 1,000 patients treated and when the background incidence (incidence in a placebo- or active-control group) approaches 0 is difficult, that difficulty will grow by orders of magnitude with aplastic anemia and SJS – TEN.

           Definitive “proof” that a drug is associated with an ADR or that a drug is not associated with some specific ADR of interest is virtually impossible given the practical limitations impacting the conduct of human RCTs when the incidence of an associated ADR is less than some 2-3% when active treatment and placebo control sample sizes are below several hundred subjects per treatment group.  For psychiatric disorders, such sample sizes or even larger sample sizes would be common with depression and anxiety disorders.  Active treatment and placebo control sample sizes can be smaller with psychotic disorders.  For example, with the development program for olanzapine for its initial indication of treatment of psychosis (later restricted to schizophrenia) the total sample sizes that allowed direct comparison with placebo were:  olanzapine – 248; placebo – 118.  Additionally, these totals were obtained in two separate RCTs.  One RCT compared placebo to olanzapine 5±2.5 mg / d, 10±2.5 mg / d, and 15±2.5 mg / d.  The other RCT compared placebo to 1 mg / d and 10 mg / d.

           Development programs in other therapeutic areas can be of much greater size.  Development programs in diabetes and cardiovascular diseases can easily exceed 5,000 and approach 10,000 subjects treated with the investigational drug.  However, complicating the matter of definitive “proof” of presence or absence of an ADR, these studies are generally conducted as drug compared to placebo as an add-on to existing therapies.  Therefore, while placebo-controlled, the ongoing treatment (or treatments) with associated ADRs can complicate definitive interpretation of safety observations.




Frey N, Jossi J, Bodmer M, Bircher A, Jick SS, Meier CR, Spoendlin J. The epidemiology of Stevens-Johnson Syndrome and toxic epidermal necrolysis in the UK. J Invest Dermatol 2017; 137: 1240-7.

International Agranulocytosis and Aplastic Anemia Study. Incidence of aplastic anemia: the relevance of diagnostic criteria. Blood 1987; 70:1718-21.

Kang YW, Ko YS, Kim KY, Sung C, Lee DH, Jeong E. Trends in health-related behaviors of Korean adults: study based on data from the 2008-2014 Community Health Surveys. Epidemiol Health. 2015 Sep 29; 37:e2015042. doi: 10.4178/epih/e2015042. eCollection 2015. 

Kaufman DW, Kelly JP, Issaragrisil S, Laporte JR, Anderson T, Levy M, Shapiro S, Young NS. Relative incidence of agranulocytosis and aplastic anemia. Am J Hematol. 2006; 81:65-7.


February 21, 2019