This week, the Annals of Internal Medicine published a summary of Penn’s development of an interactive, Web-based tool to inform our hospital planning at the early stages of this pandemic. I’d like to give you a walkthrough of how we built a model that allowed us to estimate the resources we would need to care for the surge of patients we would likely see in the coming weeks and months.

In the COVID-19 era, my colleague Scott Halpern reminded me that the most common refrain nowadays is, “All models are wrong but some are useful.” Second and only slightly less well known, is the saying, “Just because you can build a model doesn’t mean you should.” Well in this case, our health system was faced with questions like, how many beds will we need? How many ICU beds? How many ventilators? How soon will we need them?

So we needed a model to help answer these questions. But this isn’t a Kaggle tournament, or a bet against Vegas odds, so different rules apply. Decisions on resources, staffing, and ventilator machines must be made weeks in advance with layers of contingencies. We needed, still early in the epidemic in our region, to be able to explore different scenarios and understand the range of likely possibilities. It didn’t matter of the model predicted 100 or 101 beds. We needed to know if the real number was likely between X and Y, to prepare for those scenarios accordingly. The interface needed to be intuitive. And it needed to be operational yesterday. Enter an intrepid group from Penn Medicine Predictive Healthcare to make it happen.

How did we choose a modeling approach that is useful, adaptable, and readily parameterizable? At one extreme of complexity, epidemic dynamics rely heavily on the structure of contact networks, but obtaining necessary data to construct such a model is very challenging.

The Susceptible-Infected-Removed (SIR) model, based on a system of differential equations and three distinct populations, has been around for decades and is a very simplified approximation of early dynamics of an epidemic, with exponential growth. Other intermediately complex approaches include SEIR models (an adaptation of SIR with an additional population of Exposed individuals), or a polynomial curve fitting further-along epidemics and recalibrating locally, such as the IHME model.

Regardless of the underlying model, none of these approaches translate case counts into numbers of hospital or ICU beds, or ventilators, so we relied on published case series out of Italy and China (here and here) to make best guesses at how many infected patients would require hospital resources and for how long.

We supplemented these data with health system data from prior cohorts and clinical experience with patients with acute respiratory failure. But we didn’t select one number for each of our parameters. Rather, we used a range of possibilities from which we sampled to run 1,000 simulated epidemics. And one particular number is critical to the epidemic dynamics: the doubling time. That is, how long does it take for the number of cases to double?

The doubling time is tricky because it changes over the course of the epidemic as different populations are exposed, as social distancing measures are put into place, and as testing and quarantining programs are implemented. Early on we saw other countries with ranges of 2-10 days.

So we looked at extremes, best and worst cases, and a middling case to see how dramatically the doubling time would inform our hospital predictions for needed resources. The results were dramatically different based on different doubling time scenarios.

But these scenarios offered a glimpse into the ranges we were considering, rough estimates on the bounds of best and worst scenarios for the next couple of weeks. We acknowledged that the epidemic dynamics would change during that time, requiring new predictions along the way, with updated regional and local data. Hence the role for a flexible, quickly adaptable interface to explore these dynamics that could be quickly reparametrized with new data and augmented with additional layers of complexity as needed. Check out the COVID-19 Hospital Impact Model for Epidemics (CHIME) here.

From our model, we estimated the COVID-19 course in our health system would be anywhere from extremely challenging and burdensome to grim beyond imagination. Decisions to build expanded ICUs in previously non-ICU spaces, and cancel elective cases and transfers, were aided by exploring potential scenarios with CHIME.  

I am grateful to the Annals of Internal Medicine for the opportunity to share our experience and the chance to work with leaders at Penn Medicine who are responding to the pandemic with nothing but earnestness, care, grace, and tremendous compassion for our patients and health care workers.

---

The article, “Locally Informed Simulation to Predict Hospital Capacity Needs During the COVID-19 Pandemic,” was published online in the Annals of Internal Medicine on April 7, 2020. Authors are Gary E. Weissman, MD, MSHP; Andrew Crane-Droesch, PhD; Corey Chivers, PhD; ThaiBinh Luong, PhD; Asaf Hanish, MPH; Michael Z. Levy, PhD; Jason Lubken, BS; Michael Becker, BS; Michael E. Draugelis, BS; George L. Anesi, MD, MSCE, MBE; Patrick J. Brennan, MD; Jason D. Christie, MD, MSCE; C. William Hanson III, MD; Mark E. Mikkelsen, MD, MSCE; and Scott D. Halpern, MD, PhD.