Population Health

Blog Post

When Good Studies Go Negative

Lessons from a Randomized Controlled Trial of Heart Failure Patients

By:
Print:

You’ve got to ac-cent-tchu-ate the positive
E-lim-i-nate the negative
Latch on to the affirmative
Don’t mess with Mr. In-between

— Lyrics by Johnny Mercer

Recently, colleagues and I published a paper in JAMA Internal Medicine. We reported efforts to reduce readmissions or death among patients hospitalized for heart failure. Heart failure is one of the most common reasons for hospital admission, and patients with heart failure tend to get readmitted to hospitals over and over. Or they die prematurely: heart failure is a killer. Like many others, we aimed to improve these clinical outcomes.

Heart failure gets worse when there’s more fluid in the intravascular space than a weakened heart can pump. The heart can’t handle the strain, making it even harder to pump. In this vicious cycle, fluid backs up into the lungs and breathing becomes strained. The answer comes from making the heart stronger (not so easy), reducing the effort needed by reducing the volume that needs to be pumped (usually with diuretic drugs), or reducing the resistance against pumping.

We had ideas. If a big part of the problem of heart failure is that patients get overloaded with fluid, let’s find a way to measure their weight daily and alert their doctors about out-of-range weight increases. And let’s try to keep it from happening in the first place by encouraging adherence to their diuretic medication.

We gave patients scales to weigh themselves and electronic pill bottles that let us know when they were taking their diuretic medications. We set up gamified lotteries that rewarded patients with financial prizes when they adhered to the weighing and diuretic medication program. We enlisted friend and family social supporters to witness this behavior and encourage participation. We notified clinicians when weights went up or medication adherence went down so they could intervene when things went sideways.

We had a world-class team designing and overseeing the project. We enrolled over 550 patients and spent five years and several million dollars of NIH funding watching what happened.

It didn’t work. Patients had been randomized to this intervention or to usual care and it turned out the two groups were equally likely to be readmitted or die. Being inquisitive scientists, and perhaps too loyal to our original hypotheses, we tried to find some element that worked or some group where it worked better, but we couldn’t. It was a negative study and, despite our seeming cleverness, this approach failed.

What do we learn from failures like this?

One lesson is that it’s good to get your money upfront. And while that may sound like a joke, the reality is that what we were paid for was the pursuit of promise, not the promise itself. That’s how science works, and paying only to achieve desired results is a poor principle, at best, when it comes to public interest science.

The finances are different in industry, but the scientific principles remain the same. There is the old, probably apocryphal, story about Thomas Edison claiming he didn’t meet a thousand failures in making a lightbulb, but instead made a thousand discoveries of how not to make one. That story is a lot more appealing given that he eventually succeeded. Had he not, there would be no story, apocryphal or not. When it comes to this particular problem in heart failure, we haven’t yet succeeded. Eventually, we will get there, or someone else will, and whoever succeeds can look back on all the prior work and recast what seemed like failures at the time as essential steps along the path toward success.

We did learn other things, which is that patients with heart failure tend to get readmitted for lots of reasons, and heart failure itself is only one of them. So, we could be as clever as we might be about helping patients with their heart failure, but making sure their weight didn’t go up or their diuretics were taken as prescribed wouldn’t protect them from pneumonia or something else that put them back into the hospital. Patients with heart failure are sick, and sick patients are vulnerable to lots of things. Focusing on heart failure alone is too narrow. These lessons teach us to expand our approaches to anticipate more of the ways patients can need our help.

We also learned that clever incentive designs for patients may change behavior, but mostly these approaches gain yards on the field and those yards don’t turn into touchdowns every time. And communicating information to clinicians often makes them pay attention and act, sometimes in ways that make a difference, but not always, and maybe not enough to make a difference at a population level. These lessons urge us to refine and intensify our approaches to motivation, and to make sure we are directing those efforts at the patients who are likely to benefit from them.

These are hugely positive lessons emerging from a negative study. They really do lead the way forward. If there’s an overarching lesson, maybe it is this: It’s best to ask questions that are important no matter what the answer turns out to be.

The study, Remote Monitoring and Behavioral Economics in Managing Heart Failure in Patients Discharged From the Hospital: A Randomized Clinical Trial was published in JAMA Internal Medicine on May 9, 2022. The authors include David A. Asch, Andrea B. Troxel, Lee R. Goldberg, Monique S. Tanna, Shivan J. Mehta, Laurie A. Norton, Jingsan Zhu, Lauren G. Iannotte, Tamar Klaiman, Yuqing Lin, Louise B. Russell, Kevin G. Volpp.

Author

David A. Asch, MD, MBA

John Morgan Professor, Medicine and Medical Ethics and Health Policy, Perelman School of Medicine; Professor, Health Care Management, Operations, Information and Decisions, Wharton School; Executive Director, Penn Medicine Center for Innovation, Perelman School of Medicine

More from LDI

In Their Own Words

Health Care Access & Coverage | Health Equity

Building A Longitudinal Community Supports Model

Insights from Leaders of the Camden Coalition and NewCourtland Center for Transitions and Health

By:
  • Kathleen Noonan, JD, Mary Naylor, PhD, RN

Search