Physician Surplus or Shortage? Let’s Play Hard Ball
The New York Times published an editorial in 2014 proclaiming that a worsening national shortage of physicians was contributing to long waiting times for veterans and many other Americans. In response, Richard Buz Cooper, MD, noted LDI researcher and health policy contrarian who died this year, noted in his blog post “Yes, New York Times, There is a Physician Shortage” that since 2002 he had submitted three op-eds and two letters explaining the shortage. The Times declined to publish any of them, as they wrote about waste and inefficiency in healthcare, suggesting that fewer, not more, doctors were needed.
Challenging conventional wisdom
What does it take to bring new perspectives to solving persistent problems into the U.S. market-driven healthcare environment replete with competing stakeholder interests and a decentralized policy apparatus? Imagination, rigorous research, persistence, and a thick skin for absorbing all kinds of insults.
Dr. Cooper had a long, varied, and distinguished career in medicine, first as a noted hematologist oncologist who was instrumental in establishing the University of Pennsylvania’s Cancer Center, and then as dean of the Medical College of Wisconsin, preparing the next generation of physicians. In these leadership roles he had good reason to contemplate how many and what kinds doctors and other clinicians would be needed to meet 21st century healthcare opportunities and challenges. He worried about the inability of researchers and policymakers to accurately predict how many doctors, nurses, and others were needed in the near term, much less the long term. As highly touted blue ribbon panels and commissions over decades predicted shortages and surpluses in rapid succession, policy responses were quickly out of date but had long-lasting consequences, since it takes a minimum of ten years to train a new physician.
Making sense of the physician supply-demand puzzle
The latter part of Dr. Cooper’s career was devoted to conducting research to make sense out of the rapid cycle doctor shortage/surplus contentions and the ensuing national and state policy responses that seemed so out of sync with what we observed in clinical care. You might ask how Dr. Cooper went from studying physician supply and demand to publishing his posthumous book, Poverty and the Myth of Health Care Reform, which focuses on addressing poverty as the solution to the nation’s escalating medical care costs and variable patient outcomes. The answer: his quest for understanding why we could never get healthcare workforce projections right took him there.
Even before Dr. Cooper returned to Penn in 2005, he and I had connected on healthcare workforce policy research. Maybe because we both came out of clinical care disciplines, me from nursing and Cooper from medicine, we reacted with similar skepticism to conventional wisdom in healthcare management and policy circles about workforce policies and practices. In the midst of a presumed shortage of nurses, I published a paper in the New England Journal of Medicine in 1987 entitled “The Nursing Shortage: Myth or Reality.” I pointed to the shortage of hospital bedside nurses as a consequence of too few budgeted positions and hiring preferences for the least educated nurses. In a country where physicians have the longest education in the world, the notion that years of experience, not education, made the best nurses seemed bizarre.
Physician supply requirements linked to nurses
My research, and Dr. Cooper’s, led us to the same conclusions about the primary care shortage that predates Medicare. We concluded that:
- the shortage was not likely to be solved by producing more primary care physicians, since American physicians were not biting; and
- the shortage was overestimated, because non-physician providers were being prevented by irrational regulations from practicing to the full extent of their training and expertise, and because specialist physicians provided 20% or more of all primary care services as part of their ongoing roles in managing chronic and serious illnesses—evidence published by me in NEJM in 1979!
Cooper made other observations through his research, including that the U.S. did not have as many physicians to population as other comparable countries; that demand for medical care was driven by per capita GDP (wealth), which was on a growth trajectory in the U.S; and that almost all medical and technological advances were increasing demand for specialty care, which would create a shortage of specialists that no one was tracking. Indeed at the height of the push by foundations and federal policy to move to a physician workforce where primary care physicians would be in the majority (despite decades of experience suggesting it would never happen), it was highly controversial to suggest that the nation had a serious and worsening shortage of specialty physicians.
Recommendations for action
When Dr. Cooper moved back to Penn in 2005, we established and co-chaired the Council on Physician and Nurse Supply based at LDI, comprised of leading national experts in medicine and nursing, to examine evidence on the adequacy of the workforce and to make recommendations for action.The Council’s recommendations included:
- increase the supply of physicians, including specialists, by increasing residency positions;
- transition rapidly to a nurse workforce educated at least at the bachelor’s level;
- remove regulatory and payment barriers so that all health professionals practiced to the full extent of their training and expertise; and
- develop a more robust infrastructure to support clinical training for doctors, nurses, and others in a variety of clinical settings.
The Council’s recommendations on nursing are being implemented more rapidly than those on the physician supply. Thanks to a push from the Institute of Medicine, The Robert Wood Johnson Foundation, and AARP on behalf of consumers, progress is being made toward a bachelor’s qualified nurse workforce (despite stakeholder objections from community colleges) and toward full practice authority for nurse practitioners (despite objections from state medical societies). A recent report in the NEJM suggests that nurse practitioners (NPs) will likely comprise 29% of primary care practitioners by 2025 and will increase thereafter while physicians will be only 60% and decline thereafter.
The debate on the physician supply rages on. Some incremental increase in physician residencies has occurred without Medicare policy change, but vigorous debate continues on whether Medicare should reduce its current level of Graduate Medical Education support consistent with the still-dominant view that too many doctors are bad for the country. A decade ago the Council was concerned about the inadequate infrastructure to support clinical training for the future healthcare workforce. The relentless pressures on practicing physicians and NPs to do more with less, has made it even more difficult now to find clinical placements outside of hospitals for training doctors, NPs, and others. These increased demands make it difficult for community providers to accept learners who often reduce productivity, rather than improve it. Penn is leading a national demonstration testing the feasibility and effectiveness of Medicare payments to community providers to offset their lost productivity as a strategy to increase clinical training opportunities for NPs in the settings where they are most needed to increase access to primary care.
Undeterred by opposition to his ideas and insults about his research capabilities, Dr. Cooper continued to observe and question, even as he coped with his own cancer. Even though he had predicted many years ago that the boundaries of nurses would extend more and more into the traditional realms of the doctor, he was surprised that he saw the surgeon only once in the office when deciding to have surgery and that all other pre- and post-operative care was provided by nurses and a few physician assistants. And it was great care.
Buz Cooper played “hard ball” every day for decades, telling it like he saw it, making observations that made us pause, and questioning assumptions that did not jibe with his clinical experiences and common sense. Typical of his intensity and humor, he was known to say that if the association between the number of surgeons and the number of operations was due to surgeon-induced demand, what might obstetricians be up to that resulted in the birth of more babies in communities with more obstetricians? Eventually this line of thinking led him to wonder if the problem had little to do with the number of doctors and everything to do with the underlying demographics of communities, especially the uneven geographic distribution of the poor. Could their high illness burden, use of expensive healthcare, and poor outcomes be related to preexisting conditions and delayed access to healthcare? This is the line of inquiry is featured in detail in his book that might have been alternatively titled: “Finding Poverty on the Way to Solving the Physician Supply Puzzle.”