Really NEJM? Shocking Failure of the Peer Review Process for the ALBIOS Trial

The New England Journal of Medicine released three landmark trials this week.  Each trial directly addresses controversies in sepsis management that have been debated for a decade or longer: high versus low MAP goals (SEPSISPAM), a challenge to early goal-directed therapy (ProCESS), and albumin replacement (ALBIOS).

These trials will undoubtedly have a significant impact on future sepsis guidelines from SCCM, which is incredibly exciting.  Perhaps even more exciting is to see the dramatic improvement in early sepsis mortality between Rivers in 2001 (30.5 to 46.5%) and ProCESS (18.2 to 21%).

In pouring through each of these trials to provide a timely update to the ICU Trials mobile application, I was absolutely shocked by the failure of the peer review process in the ALBIOS trial.

Peer Review Gone Wrong

Briefly, the ALBIOS trial evaluated whether daily repletion with 20% albumin in hypoalbuminemic septic patients would improve patient outcomes compared to no albumin repletion.  The trial found no difference in 28-day mortality, 90-day mortality, or any other relevant clinical endpoints.  In short, this trial is yet another trial supporting the slow death of using albumin for a replacement or resuscitation fluid in the critically ill patient (see the SAFE trial).c

Despite an overall negative trial, the manuscript has the stench of determined, obstinate, unyielding bias in favor of finding a benefit to albumin at all cost.  In theory, the peer review process is designed to temper clear authorship bias in the face of negative results.  This was simply not the case in the ALBIOS trial.

Post-Hoc Subgroup Analysis of a Secondary Endpoint

In an effort to find a clinical benefit to albumin at all costs, the authors conducted a post-hoc (unplanned) subgroup analysis of only septic shock patients.  Lo and behold, the statistical heavens opened and the authors report that “a significant difference was observed” (not actually stating that the difference was the secondary endpoint of 90-day mortality) with a relative risk of 0.87 (95% CI 0.77-0.99).  The authors defend their nearly non-significant confidence interval by concluding that “adjustments for baseline covariates did not significantly modify these results” with a casual reference to the 27-page supplementary appendix.

Within the supplement (Table S6), there is a numerically lower number of deaths in the albumin replacement group (43.6% vs 49.9%).  After adjusting for “unbalances” (variables with p<0.05 at baseline), the mortality benefit still holds true.  However, a third adjustment for “clinically relevant variables” (age, baseline SOFA, baseline lactate, baseline SCvO2) demonstrates no significant difference in mortality: RR 0.88 (95% CI 0.77-1.01), p=0.07.

At this point, a peer reviewer or a critical reader of NEJM should be asking the following questions:

1. Why was a post-hoc analysis of 28-day mortality among septic shock patients not performed? (hint: it probably wasn’t statistically significant)
2. Why did the manuscript fail to report the second, “clinically relevant” adjustment for 90-day mortality? (hint: conflicting statistical tests don’t lead to a sexy, controversial conclusion)

To the credit of NEJM, I would like to think that the second, “clinically relevant” adjustment was requested by the peer review process.  After all, based on the author’s comments regarding this post-hoc analysis, it seems unlikely that they would willingly publish this second, conflicting adjusted relative risk.

An Even Greater Failure

I completely understand the attractiveness of data dredging in the hopes of finding an interesting subgroup or significant clinical endpoint to help prompt future research efforts.  There is a difference, however, in a hypothesis-generating post-hoc analysis and simply making broad, unfettered clinical conclusions.

The ALBIOS authors conclude the manuscript with the following statements:

Post hoc univariate and multivariate analyses of data from the 1121 patients with septic shock showed significantly lower mortality at 90 days in the albumin group than in the crystalloid group

As discussed above, the “significant” benefit was only seen in one of the multivariate analyses, which of course is not mentioned at all within the manuscript.  Furthermore, there is no mention of a similar post-hoc analysis for the primary endpoint of 28-day mortality.

This analysis was not prespecified, and therefore it may be characterized by well-known biases. Nonetheless, a state of shock associated with severe sepsis represents a well-defined clinical entity.

The authors acknowledge the potential bias in a post-hoc subgroup analysis of a secondary endpoint, but quickly diminish the bias by arguing that septic shock is a reasonable subgroup to have studied.  Of course, the most reasonable approach would have been to pre-specify septic shock as a secondary analysis, but hindsight must be 20/20.

The clinical benefit of albumin that was seen in the post hoc analysis of the subgroup of patients with septic shock warrants further confirmation.

The final sentence of the manuscript (above) displays the outright bias of the authors and the failure of the peer review process to keep the authors honest.  The idea of “further confirmation” suggests that the ALBIOS study confirms the benefit of albumin replacement in septic shock, which was simply not the case.

At best, this post-hoc, data-dredging analysis is hypothesis generating.  It makes for a great discussion during a journal club, but is not appropriate to suggest a change in clinical practice may be warranted.  At worst, this represents a type I statistical error that has been misrepresented in one of the most reputable medical journals that will lead to the prolonged, painful death knell of albumin replacement in the critically ill patient.

Photo by Ribarnica, Tom Brown