A new analysis from clinical epidemiologist Dr. Richard Lilford of the U.K.’s University of Birmingham and critical care physician Dr. Peter Provonost of Johns Hopkins University School of Medicine suggests that death rates are a poor measure of the quality of care at a hospital. Writing in the British Medical Journal Lilford and Provonost argue that relying on a standardized mortality ratio—a measure currently used to assess hospital quality that is meant to distinguish preventable deaths—to assess quality is short-sighted and inaccurate, and belief that adjusting for risk will accurately distinguish which deaths are preventable from those that are inevitable is misguided.
The authors break down the specific shortcomings of death rates as measures of quality—pointing out that risk adjustment measures can exaggerate instead of control for certain factors, for example, and highlighting previous research showing that “quality of care accounts for only a small proportion of the observed variance in mortality between hospitals.” What’s more, they point out, a review of previous studies into the subject found little correlation between standardized mortality ratios and actual quality of care. Put simply, the authors conclude, ‘hospital mortality rates are a poor diagnostic test for quality and [standardized mortality ratios] do not identify preventable deaths.”
Lilford and Provonost are careful to point out that monitoring hospital quality is important and essential, but argue that, instead of using a crude measure of mortality rates, hospitals should instead audit treatment processes. If it is revealed that one hospital isn’t consistently giving a certain medication to patients before hip replacement surgery, for example, it is clear what action they need to take to improve that particular measure. (Audit and feedback processes have dramatically improved treatment for heart attack patients in England, the authors point out, to the point that, now, in almost 100% of cases heart attack patients are quickly given treatment to unblock the coronary artery).
Last year, high death rates at Mid Staffordshire Hospital in England prompted denunciations for “appalling care” and an inquiry by the National Health Service, the BBC reports. Media reports said that as many as 400 people may have died unnecessarily at the hospital due to poor quality care. Yet, those alarming claims aren’t based on good data, Lilford and Provonost argue. “Given the fragility of [standardized mortality ratios], they should not be used to calculate excess deaths resulting from poor care, yet Mid Staffordshire hospital was blamed for 400 excess deaths on this precarious basis,” they write.
In an accompanying editorial, Nick Black, a professor of health services research
at the London School of Hygiene and Tropical Medicine echoes the notion that death rates are poor indicator of hospital quality. Black writes: “A consequence of a failure to provide alternative forms of care has been that hospitals have taken on the role of providing a place for people to die. About half of us will end our days in a hospital bed. This makes it perverse to use a hospital’s mortality statistics to judge its quality of care, given that deaths are often an expected and accepted outcome.”