Jeffrey Silber Team Develops New Hospital Ranking System

Jeffrey Silber Team Develops New Hospital Ranking System

Uses Template System and Multivariate Matching
In this 3-minute LDI excerpt of a longer Knowledge@Wharton video interview, Jeffrey Silber explains the new hospital care ranking system developed by a team he led.

A study led by LDI Senior Fellow Jeffrey Silber and co-authored by Senior Fellows Paul Rosenbaum, Lee Fleisher and Orit Even-Shoshan that has created a new way to rank hospital performance is the subject of a major Knowledge@Wharton article.

Silber explained the new methodology uses a template system and multivariate matching to characterize patient populations in a manner that makes rankings more truly "apples to apples" comparisons. He notes that one of the greatest frustrations of chief medical officers is to read media stories that their institution has been ranked very low in some quality care report. Many dismiss such reports by claiming that their patient population is substantially sicker than those of the higher ranked institutions.

'Indirect standarization'
Silber said the media's annual "report card" ranking of hospitals is based on "indirect standardization" -- an estimating method that looks at different types of patients at different hospitals -- even non-overlapping populations -- and extrapolates conclusions about hospital quality.

"It's not really a fair way to compare hospitals," Silber told Knowledge@Wharton, "because one hospital might have an easier set of patients [and] another hospital might have a much different, more difficult set of patients."

To make this process more accurate, Silber's team used a "direct standardization" approach that matched 300 general and orthopedic surgery patients in each of 217 hospitals to a template of hundreds of characteristics.

"What we show in the paper," Silber said, "is that these 300 patients at each and every hospital are incredibly similar. Their age is the same. Their rate of diabetes and heart failure and all the characteristics we’d be interested in are incredibly similar, statistically undifferentiable to the other hospitals. Yet what we find is that the outcomes are very, very different."

'Stamped to a template'
"We showed," he continued, "that you can get very close matches across hospitals stamped to this template of patients [and] that there is great variability in the way the hospitals handled the patients and the outcomes of the patients."

"I think seeing that the matches were so close and, at the same time, seeing that the outcomes were so different was something that I think people should realize — there are better ways to measure them than what we’ve been doing up to now," Silber said.

Silber also said this new system could be easily applied to nursing homes and schools as well as hospitals.

"What we're doing here is take from one field -- multivariate matching for which Paul Rosenbaum is the world's expert -- and applying it to quality assessment. I just don't think that's been done and it certainly hasn't been done with a template which levels the playing field and lets you really see if your quality is different from others."

Silber, MD, PhD, is Chair in Health Services Research and Director of the Center for Outcomes Research at The Children’s Hospital of Philadelphia (CHOP). He is also a Professor of Pediatrics, Health Care Management, and Anesthesiology & Critical Care at Penn's Perelman School of Medicine.

Rosenbaum, PhD, AM, is a Professor of Statistics at the Perelman School of Medicine; Fleisher, MD, is Chair of Anesthesiology and Professor of Anesthesiology and Critical Care at Penn's Perelman School of Medicine; Even-Shoshan, MS, is Associate Director of CHOP's Center for Outcomes Research.