Wide Press Pickup For Penn Yelp Hospital Ratings Study

Wide Press Pickup For Penn Yelp Hospital Ratings Study

Real-Time Patient Social Media Reviews vs. Federal Quality Survey Reports

A University of Pennsylvania study led by Raina Merchant  that compared real-time Yelp patient hospital reviews against federal hospital patient satisfaction reports has garnered widespread media pickup since its publication in Health Affairs last week.

The study's most striking finding is that the federal government's method for tracking patients' perceptions about the quality of their hospital experiences -- the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) -- lacks  twelve categories of common patient concerns found on Yelp. These include things like "compassion of staff," "quality of nursing" and "billing department interactions." 


Photo: Allan Hunter Shoemaker

Raina Merchant, MD,  Director of the University of Pennsylvania Social Media and Health Innovation Lab and LDI Senior Fellow.

'Missing the major drivers'
Penn researchers concluded that, "current hospital ratings based on HCAHPS may be missing the major drivers of patients' overall experiences of care... Yelp topics may provide a more nuanced view of aspects of hospital quality that patients value."

Funded by the Agency for Healthcare Research and Quality (AHRQ), the HCAHPS survey is conducted for the Centers for Medicare and Medicaid Services (CMS) and is a key part  of the growing regulatory emphasis on patient satisfaction as a metric of hospital quality.

The study has been the subject of major articles in mainstream media outlets including the Washington Post, NRP's NewsworksBecker's Hospital ReviewModern HealthcareHealthcare IT NewsHealthcare Finance NewsMedscapeSTAT News,  FierceHealthCarePhiladelphia Magazine, and the Philadelphia Business Journal.

Headquartered in San Francisco, Yelp is a social media platform specializing in crowd-sourced reviews of businesses of all types in local markets across the country. Its online database contains nearly 100 million reviews and logs 142 million visits a month.

5 star rating system
As with restaurants, hotels, beauty shops and other local businesses, consumers can rate local hospitals according to a Yelp five star system accompanied by a narrative about their own personal experiences and observations.

Penn researchers analyzed HCAHPS and Yelp ratings for 1,352 hospitals whose services were the subject of 16,862 Yelp reviews.

 Included in the study paper are actual excerpts from Yelp narratives ranging from angry critiques to warm compliments about specific hospitals. Here are two samples:

1.) "I asked a question... [the nurse] snapped at me with a raised voice and very condescendingly asked me if there was anything else I did not understand. No one deserves to be treated like this for a question that is asked to their health care PROFESSIONAL."

2. ) "My wife gave birth here and the attention and service was excellent..."

Merchant, MD, an Assistant Professor of Emergency Medicine at Penn's Perelman School of Medicine and an LDI Senior Fellow, is also Director of Penn's Social Media and Health Care Innovation Lab. Since its launch in 2013, the Lab has been pursuing a variety of studies exploring how to extract scientifically valid health data from social media streams.

Also members of the research team were LDI Senior Fellows Rachael Werner, MD, PhD; Zachary Meisel, MD; and David Asch, MD, MBA.