When evaluating a clinical trial, readers often jump to the P value of the primary endpoint to determine whether the results of a trial are “statistically significant” or not. Although the P value is truly a continuous variable, the scientific community has been conditioned to disregard all results with P values ≥ 0.05, but to fully endorse any trials with a “statistically significant” P value less than 0.05.
Putting the debate and controversy about P values aside for the moment, as a reader, would you be less impressed with a study that changed from being statistically significant to insignificant if one single patient changed from not having the primary endpoint to having the primary endpoint? Especially in an era with a blind reliance on P values, the knowledge of the “fragility” or “robustness” of a study’s P value is another useful data point for readers to critically understand and analyze the results of a clinical trial.
The Concept of the “Fragility Index” for Clinical Trials
The New England Journal of Medicine released three landmark trials this week. Each trial directly addresses controversies in sepsis management that have been debated for a decade or longer: high versus low MAP goals (SEPSISPAM), a challenge to early goal-directed therapy (ProCESS), and albumin replacement (ALBIOS).
These trials will undoubtedly have a significant impact on future sepsis guidelines from SCCM, which is incredibly exciting. Perhaps even more exciting is to see the dramatic improvement in early sepsis mortality between Rivers in 2001 (30.5 to 46.5%) and ProCESS (18.2 to 21%).
In pouring through each of these trials to provide a timely update to the ICU Trials mobile application, I was absolutely shocked by the failure of the peer review process in the ALBIOS trial.
Summary of the Kumar 2006 Trial
Among patients with septic shock, every hour of delay in appropriate antibiotic administration was associated with a significant increase in mortality.
Summary of the CORTICUS Trial (Sprung 2008)
Hydrocortisone therapy did not improve outcomes among patients with septic shock (onset within 72 hours), although it did shorten the duration of vasopressor dependence.
Summary of the Annane 2002 Trial
Among patients with very early septic shock who were non-responders to a cosyntropin stim test, hydrocortisone/fludrocortisone therapy improved 28-day survival. Furthermore, steroid therapy reduced duration of vasopressor therapy in all patients (regardless of stim test response).
Summary of the NICE-SUGAR Trial
Among critically ill patients, intensive glucose control increased 90-day mortality and the incidence of severe hypoglycemia compared to conventional therapy.
Summary of the Leuven I (van den Berghe 2002) Trial
In surgical ICU patients (primarily cardiac), intensive insulin therapy reduced ICU mortality, renal impairment, and bloodstream infections. The rate of severe hypoglycemia was higher with intensive insulin.
Summary of the NINDS Trial
In patients presenting within 3 hours of ischemic stroke, alteplase improved 3-month neurological function (NNT=9) but did not impact 24-hour symptoms or mortality. In patients receiving alteplase, approximately one-quarter had minor bleeding, and 6.4% had symptomatic ICH (NNH=17).
Summary of the Kress 2000 Trial
Medical ICU patients receiving continuous infusion sedation with daily interruption were liberated from mechanical ventilation and left the ICU quicker, but this effect did not translate to a shorter hospital course or a mortality benefit.
Summary of the EPaNIC (Casaer 2011) Trial
Early initiation of TPN increased ICU and hospital stay, the incidence of infection, and total healthcare costs. Delaying parenteral nutrition up to 7 days had no effect on mortality.