Really NEJM? Shocking Failure of the Peer Review Process for the ALBIOS Trial

Three nuts waiting to be cracked

The New England Journal of Medicine released three landmark trials this week.  Each trial directly addresses controversies in sepsis management that have been debated for a decade or longer: high versus low MAP goals (SEPSISPAM), a challenge to early goal-directed therapy (ProCESS), and albumin replacement (ALBIOS).

These trials will undoubtedly have a significant impact on future sepsis guidelines from SCCM, which is incredibly exciting.  Perhaps even more exciting is to see the dramatic improvement in early sepsis mortality between Rivers in 2001 (30.5 to 46.5%) and ProCESS (18.2 to 21%).

In pouring through each of these trials to provide a timely update to the ICU Trials mobile application, I was absolutely shocked by the failure of the peer review process in the ALBIOS trial.

Peer Review Gone Wrong

Briefly, the ALBIOS trial evaluated whether daily repletion with 20% albumin in hypoalbuminemic septic patients would improve patient outcomes compared to no albumin repletion.  The trial found no difference in 28-day mortality, 90-day mortality, or any other relevant clinical endpoints.  In short, this trial is yet another trial supporting the slow death of using albumin for a replacement or resuscitation fluid in the critically ill patient (see the SAFE trial).c

Despite an overall negative trial, the manuscript has the stench of determined, obstinate, unyielding bias in favor of finding a benefit to albumin at all cost.  In theory, the peer review process is designed to temper clear authorship bias in the face of negative results.  This was simply not the case in the ALBIOS trial.

Post-Hoc Subgroup Analysis of a Secondary EndpointStatistical Trickery

In an effort to find a clinical benefit to albumin at all costs, the authors conducted a post-hoc (unplanned) subgroup analysis of only septic shock patients.  Lo and behold, the statistical heavens opened and the authors report that “a significant difference was observed” (not actually stating that the difference was the secondary endpoint of 90-day mortality) with a relative risk of 0.87 (95% CI 0.77-0.99).  The authors defend their nearly non-significant confidence interval by concluding that “adjustments for baseline covariates did not significantly modify these results” with a casual reference to the 27-page supplementary appendix.

Within the supplement (Table S6), there is a numerically lower number of deaths in the albumin replacement group (43.6% vs 49.9%).  After adjusting for “unbalances” (variables with p<0.05 at baseline), the mortality benefit still holds true.  However, a third adjustment for “clinically relevant variables” (age, baseline SOFA, baseline lactate, baseline SCvO2) demonstrates no significant difference in mortality: RR 0.88 (95% CI 0.77-1.01), p=0.07.

At this point, a peer reviewer or a critical reader of NEJM should be asking the following questions:

  1. Why was a post-hoc analysis of 28-day mortality among septic shock patients not performed? (hint: it probably wasn’t statistically significant)
  2. Why did the manuscript fail to report the second, “clinically relevant” adjustment for 90-day mortality? (hint: conflicting statistical tests don’t lead to a sexy, controversial conclusion)

To the credit of NEJM, I would like to think that the second, “clinically relevant” adjustment was requested by the peer review process.  After all, based on the author’s comments regarding this post-hoc analysis, it seems unlikely that they would willingly publish this second, conflicting adjusted relative risk.

An Even Greater Failure

I completely understand the attractiveness of data dredging in the hopes of finding an interesting subgroup or significant clinical endpoint to help prompt future research efforts.  There is a difference, however, in a hypothesis-generating post-hoc analysis and simply making broad, unfettered clinical conclusions.

The ALBIOS authors conclude the manuscript with the following statements:

Post hoc univariate and multivariate analyses of data from the 1121 patients with septic shock showed significantly lower mortality at 90 days in the albumin group than in the crystalloid group

As discussed above, the “significant” benefit was only seen in one of the multivariate analyses, which of course is not mentioned at all within the manuscript.  Furthermore, there is no mention of a similar post-hoc analysis for the primary endpoint of 28-day mortality.

This analysis was not prespecified, and therefore it may be characterized by well-known biases. Nonetheless, a state of shock associated with severe sepsis represents a well-defined clinical entity.

The authors acknowledge the potential bias in a post-hoc subgroup analysis of a secondary endpoint, but quickly diminish the bias by arguing that septic shock is a reasonable subgroup to have studied.  Of course, the most reasonable approach would have been to pre-specify septic shock as a secondary analysis, but hindsight must be 20/20.

The clinical benefit of albumin that was seen in the post hoc analysis of the subgroup of patients with septic shock warrants further confirmation.

The final sentence of the manuscript (above) displays the outright bias of the authors and the failure of the peer review process to keep the authors honest.  The idea of “further confirmation” suggests that the ALBIOS study confirms the benefit of albumin replacement in septic shock, which was simply not the case.

At best, this post-hoc, data-dredging analysis is hypothesis generating.  It makes for a great discussion during a journal club, but is not appropriate to suggest a change in clinical practice may be warranted.  At worst, this represents a type I statistical error that has been misrepresented in one of the most reputable medical journals that will lead to the prolonged, painful death knell of albumin replacement in the critically ill patient.

Photo by Ribarnica, Tom Brown

4 thoughts on “Really NEJM? Shocking Failure of the Peer Review Process for the ALBIOS Trial



  2. Steve R.


    I read your letter in Critical Care and thought it was an excellent synopsis of the issues surrounding the validity of the trial’s conclusions. The letter also managed to lay these out without being confrontational. I found the authors’ response to be totally inappropriate and reflect poorly upon them as researchers. It was no doubt a significant effort to conduct this trial, but for researchers that claim to care about the advancement of medical science, they either 1) didn’t understand, or 2) didn’t care to address your major points. They then reiterated the “further confirmation” statement as a jab.

    The aggressiveness and condescension of their response makes me trust their results even less, and leads me to believe that they had a point to prove, and were willing to bend the data to prove it. It also makes me wonder about the caliber of character of the kind of people that conduct these studies. It was absolutely shocking to me that they were so unwilling to discuss their data in an objective way. Why conduct the research if you don’t care about the science or the validity of the results? I have a feeling I know the answer…

    1. Sean P. Kane Post author

      Steve — thanks for the kind feedback. I couldn’t agree more. Although I felt like the letter fell on deaf ears for the ALBIOS authors, I think other readers were able to at least raise a critical eyebrow.

  3. Gary Cowen

    I found your analysis interesting. I am disturbed by the larger issue of inadequate peer review by the NEJM. For example, there have been a series of papers over the past few months regarding the ACA. There appears to be bias in favor of the law. A recent paper discussing high deductible health plans seemed to downplay their overall effect. Bottom line, if your plan requires you to pay the first $3000 to $10,000 out of pocket, there seems little incentive to seek care of chronic conditions or even visit the doctor unless you earn substantially above the average. While time will tell regarding the effect of these plans, there does not seem to be vigorous debate in the Journal. I would welcome that debate.


Leave a Reply

Your email address will not be published. Required fields are marked *

Note: is intended for healthcare providers. Comments regarding personal or patient-specific medical questions or advice will be removed.