Data Integrity in Real-World Evidence

Bobby Yeh, MD, from Beth Israel Deaconess Medical Center in Boston, discusses data integrity in real-world evidence—an important topic for clinicians who are both consumers and producers of clinical literature. Dr. Yeh gave this presentation at a Coronary Artery and Myocardial Protected PCI (CAMP PCI) educational event on cardiogenic shock and high-risk PCI in June 2022.

“How do we know what we know, in general in medicine, and in interventional cardiology?” Dr. Yeh asks, explaining that every day in clinical practice clinicians are asked to make treatment decisions in the absence of evidence.

For example, Dr. Yeh describes two studies that the same journal published within a month. The first concluded that closure devices are associated with a 9% relative reduction in mortality. The second found that the use of closure devices was associated with no reduction in mortality. With very opposite conclusions and no reconciliation of these different results, Dr. Yeh asks, “What are we supposed to make of this when even the editors of journals sort of can’t make heads or tails of what is a good analysis and what isn’t a good analysis?”

Dr. Yeh explains that the use of controls is a fundamental component of basic science, and he advises clinicians when consuming or producing literature, to think about whether observational experiments have been controlled. He describes an analysis he’s working on with the FDA of more than 23,000 Medicare beneficiaries undergoing percutaneous coronary intervention who presented with acute myocardial infarction complicated by cardiogenic shock. “Depending on how we analyze the data, we can find a bunch of different things,” Yeh notes. One type of analysis can show that Impella® heart pump or ECMO is associated with harm in these patients, and another can show no association with harm. He states that the conclusion of this analysis was that the empirical evidence and background knowledge suggest that violations of the assumptions for each of the methods used are likely, lowering confidence in clinical interpretation of the observed associations as causal effects.

Dr. Yeh concludes his presentation by discussing the challenges of enrolling patients in randomized clinical trials, noting that clinical trials, such as PROTECT IV and RECOVER IV, often challenge a clinician’s standards of care and can be difficult to conduct. Yet, he emphasizes that trials that challenge standards of care are needed. “You can ruin the results of a clinical trial based on who we enroll and who we don’t enroll. And that’s particularly true if we are holding out the patients out of those trials for whom we think that they’re the most likely to benefit.”