Health Policy$ense

What Big Data Can’t Tell Us, But Kolstad’s Paper Suggests

A New Study of Surgeons' Quality 'Report Cards'

The good news is that LDI Senior Fellow Jonathan Kolstad has won the prestigious Arrow Award for the Best Paper in Health Economics for his study entitled, Information and Quality When Motivation is Intrinsic: Evidence from Surgeon Report Cards. The even better news is what his study found:  empirical evidence that physicians (well, surgeons in particular) respond to incentives to do good far more than to incentives to make money.

Kolstad’s paper is not an easy read, but quite an important one.  He analyzed surgeons’ response to the introduction of quality “report cards” for cardiac surgery in Pennsylvania. His new contribution was to model response based on information only the surgeon would obtain from the report card (performance relative to peers), which  was unrelated to patient demand.  He compared that to responses based on shifts in patient demand, which would directly affect profits.  Looking at the pre- and post-reporting period, Kolstad found that the intrinsic incentives to perform well provoked a response in quality measures (statewide risk-adjusted mortality rate) four times greater than the response to changes in patient demand.  Thus, doing good (or at least as well as one’s peers) motivated surgeons to improve quality much more than the desire to maintain or expand business and profits.  He concludes:

The risk-adjustment model that underlies quality report cards provides a simple way of identifying the magnitude of the new information provided to surgeons and its effect on performance. Surgeons who gain more information about their performance relative to their peers (from risk adjustment) improve significantly more. A structural patient choice model allows me to estimate the profit incentives from quality reporting. Conditioning on extrinsic incentives, the intrinsic response to information leads to significant declines in RAMR and is large relative to the response to profit motives.

In a thoughtful, engaging New Yorker post today, LDI Senior Fellow Lisa Rosenbaum discusses these results in the context of Medicare big “data dump” of physician payments.  In it, she notes that the "big data" reveal nothing about quality of care, and may serve to reinforce the public’s view of greedy physicians as the problem in American health care:

That’s why asking the public to use this information to identify waste belies the complexity of physician decision-making. Do physicians respond to financial incentives? Yes. Should we tolerate care that offers patients no benefit? Absolutely not. But are profit motives the primary drivers of physician behavior? My own sense is that most physicians are primarily motivated by trying to do the right thing for their patients. Combing through these data, however, creates the impression that the pecuniary trumps the humane. What else can one conclude from information that only tells you how much physicians do and what they bill?

Go read her entire post, “What Big Data Can’t Tell Us About Health Care.”  Right away.