What makes something a “landmark” critical care trial?

I receive quite a bit of feedback regarding one of my more popular mobile applications, ICU Trials by ClinCalc.  Often, these emails involve a request to add a particular “landmark” trial with the next application update.  With 10-20 recent requests for new trial additions, I’ve been forced to re-evaluate my criteria for what constitutes a seminal paper in the area of critical care medicine.

Although there isn’t an official scoring system or absolute list of must-have criteria, I’ve developed a simple list of ten important factors that help differentiate important trials from paramount, landmark gems in evidence-based medicine.

1. Article Age

In baseball, players must be retired for at least five years before they are eligible for the Baseball Hall of Fame.  This “cooling off” period is essential for overcoming the excitement of a recent season and evaluating the player more fairly.

Similarly, clinical trials can often be exciting when first published, but their clinical relevance and importance in practice can quickly fade over time.  As an example, the recently published 6S and CHEST trials, which evaluated hydroxyethyl starch (HES) in severe sepsis and ICU patients, have high clinical applicability but will likely decline over time as the excitement of HES as a resuscitative fluid declines.  This “waning relevancy effect” of young article age is massively compounded if the results are negative (see criterion #4).

For these reasons, with some important exceptions, I am often slow to adopt newly published articles into the database until they have, like a fine wine, aged with time.

2. Mentioned in Guidelines and Reading Lists

By definition, a pivotal clinical trial should have a sustained, important impact on clinical practice.  This effect is best seen when a landmark trial makes its way into evidence-based clinical practice guidelines.  In my mind, there’s no better case for classifying a trial as “landmark” if a multidisciplinary panel of experts references such a trial as a reason for a recommendation.

In a similar vein, the mention of a trial in the reading list of well-respected circles, such as the American Thoracic Society, also produces a strong argument.

3. Maintains Long-Term Relevancy

Closely related to article age (criterion #1), a landmark trial must maintain long-term relevance in clinical practice.  With younger trials, this effect may be difficult to predict without a crystal ball.  In trials published years or decades ago, this effect is apparent in present day and requires no prognostication.

4. The Issue of Negative Trials

Negative trials are a tricky beast.  In many instances, when a trial shows no benefit (or even harm) with an intervention, the results are profoundly relevant when first published, but its impact often dwindles quickly with time.  The dwindling impact of these trials is more commonly seen in cases where the treatment was unproven or less commonplace before the trial’s publication.  An excellent example of this are the two recently-published trials regarding high-frequency oscillation in ARDS: OSCAR and OSCILLATE.  While these trials have high clinical relevancy, the idea of high-frequency oscillation will rapidly decline and the findings will become historically interesting but less clinically relevant.

There are, of course, a few exceptions to the idea that negative trials don’t have the same staying power as a positive counterpart.  Usually negative trials become seminal papers because the intervention was widespread, recommended by experts, or controversial before the negative publication.  Important examples of landmark negative trials in critical care include ANZICS Dopamine, PROWESS SHOCK, and NICE SUGAR, just to name a few.

5. Controversial Appeal

Controversial, conflicting trials lead to increased clinical interest and debate.  On issues like drotrecogin alfa (Xigris), tight glycemic control, and corticosteroids for ARDS or septic shock, the fact that we have conflicting data makes these topics all the rage.

With controversial issues, we have editorials, speakers at national conventions, and opinionated healthcare providers who have deep-seeded beliefs for one side of an argument.  By virtue of the debate, controversial trials often become etched in our minds as landmark trials.

6. Significant Beneficial Effect

Some trials, such as the Early Goal Directed Therapy study (Rivers 2001), become landmark studies because the benefit demonstrated is so important that it’s difficult to ignore.  Clinicians are forced to pay attention when a trial shows a 34% relative risk reduction in 28-day mortality.  Best of all, these studies become controversial (criterion #5) if they show a significant benefit but have important methodological flaws and limitations, such as the Meduri papers.

7. Conflicting Evidence

Should we use steroids in sepsis?  Annane 2002 says yes, but CORTICUS says no.  Conflicting evidence leads to controversial appeal (criterion #5) and an important, in-depth evaluation of the literature.

8. Slaughtering the Sacred Cow

Trials can quickly become landmark when they threaten the conventional teaching and wisdom of current practice.  Prior to the publication of NICE SUGAR, extremely tight glycemic control was standard of care.  It was a well-accepted fact that tight glycemic control (80-110 mg/dL) improved outcomes, particularly in surgical ICU patients.  In one fell swoop, NICE SUGAR challenged our thinking about tight glycemic control, which makes it such a pivotal trial in critical care medicine.

9. Journal or Author/Trial Group

Although not as important as other factors, the journal and authors (or trial group) may play a role in determine the impact of a trial.  Some names, such as Meduri and Annane, carry significant weight in the critical care world and have a much higher chance of being recognized.  Similarly, core clinical journals like NEJM, JAMA, and Crit Care Med hold themselves to a high standard of article quality.

A wonderful example is the initial evidence supporting the use of carbapenems for prophylaxis in acute pancreatitis.  At one time, this was a well-accepted practice as part of routine pancreatitis management.  Aside from subsequent negative trials being published, one of the reasons for this practice not being as well received is due to the less-relevant journal it was published in — Surgery, Gynecology & Obstetrics.

10. Commonly Cited and Discussed

Finally, the most basic criterion for determining whether a trial becomes “landmark” is how it impacts the critical care community.  Is the article frequently cited in literature, discussed in meetings, and printed for debate on medical rounds?  In the end, the community itself determines how important and pivotal a given trial is.

 

What criteria make a trial a pivotal, landmark, seminal trial in the evidence-based medicine world?  I would love to hear your thoughts by contacting me or leaving a comment below.

Photo by kern.justin