Quality Measures: An SEC for Health Care?

If you have ever tried to choose a physician or hospital based on publicly available performance measures, you may have felt overwhelmed and confused by what you found online. The Centers for Medicare and Medicaid Services, the Agency for Healthcare Research and Quality, the Joint Commission, the Leapfrog Group, and the National Committee for Quality Assurance, as well as most states and for-profit companies such as Healthgrades and U.S. News and World Report, all offer various measures, ratings, rankings and report cards. Hospitals are even generating their own measures and posting their performance on their websites, typically without validation of their methodology or data.

The value and validity of these measures varies greatly, though their accuracy is rarely publically reported.  Even when methodologies are transparent, clinicians, insurers, government agencies and others frequently disagree on whether a measure accurately indicates the quality of care. Some companies’ methods are proprietary and, unlike many other publicly available measures, have not been reviewed by the National Quality Forum, a public-private organization that endorses quality measures.

Depending where you look, you often get a different story about the quality of care at a given institution. For example, none of the 17 hospitals listed in U.S. News and World Report’s “Best Hospitals Honor Roll” were identified by the Joint Commission as top performers in its 2010 list of institutions that received a composite score of at least 95 percent on key process measures. In a recent policy paper, Robert Berenson, a fellow at the Urban Institute, Harlan Krumholz, of the Yale-New Haven Hospital Center for Outcomes Research and Evaluation, and I called for dramatic change in measurement.  (Thanks to The Health Care Blog for highlighting this analysis recently.)

We made several recommendations, including focusing more on measuring outcomes such as mortality and infections rather than processes (e.g. whether patients received the recommended treatment) or structures of care (e.g. whether ICUs are staffed around the clock with critical care specialists). We urged that measures be at the organization level rather than clinician level, to reflect the fact that safety and quality are as much products of care delivery systems as of individual clinicians. We propose investments in the “basic science” of measurement so that we better understand how to design good measures. You can read these and other recommendations in the analysis.

Of the proposals, perhaps the biggest game-changer would be the creation of an entity to serve as the health care equivalent of the U.S. Securities and Exchange Commission. Rather than wading through a bevy of competing and often contradictory measures, patients and others would have one source of quality data that has national consensus behind it. We write:

“Under this model, this entity would set the rules for the development of measures and the transparent reporting of performance of these measures, analyze progress (with input from clinicians, patients, employers, and insurers), and audit publicly-reported quality measure data. Private sector information brokers could then conduct secondary analyses of the reports, much like happens in the financial industry through companies like Bloomberg. This SEC-like model would thus ensure that all publicly-reported quality measure data are generated from a common basis in fact and allow apples-to-apples comparisons across provider organizations.”

Before the SEC was created, in the aftermath of the Wall Street Crash of 1929, information provided by one business typically could not be compared to another, as there were no common standards for reporting financial performance. It’s more than 80 years since then, and health care is stuck in a similar situation, despite great efforts to create measures to drive improvement and inform patients’ decisions. It’s time that we catch up. A SEC-like entity could have private sector rule-setting, public sector auditing and transparency, and private sector reanalysis, working from a common book of truth.

Advancing the science of measurement is one of three content tracks in Johns Hopkins’ first Forum on Emerging Topics in Patient Safety, to be held Sept. 23-25 in Baltimore. Experts from a wide range of backgrounds will gather to help generate ideas around this crucial issue. Among the speakers on this track are Patrick Conway, Chief Medical Officer for the Centers for Medicare and Medicaid Services; John Santa, Director of the Consumer Reports Health Ratings Center; Niek Klazinga, Coordinator of the Health Care Quality Indicator Project at the Organisation for Economic Co-operation and Development; and Robert Berenson of the Urban Institute. Aimee Guidera, founder of the Data Quality Campaign, which has encouraged the creation and use of high-quality data in education, will provide perspectives from her field that may translate to health care. If you are interested in this topic and would like to contribute to the recommendations that come from the forum, please join us in September.