As standardized exam scores increasingly define success for students, teachers and schools, parents worry about the dangers of “teaching to the test”—and of their children being judged by tests with low or unknown validity. We want our children to perform well on tests, of course, yet only if they measure something that students, patients and teachers believe really matter. We also want the education system to inspire students develop into well-rounded people, not just skilled exam-takers.
In health care there is a similar danger of focusing on improving our “test scores” at the expense of real improvement in patient safety—and in this case, the exams have serious flaws. The federal government uses a composite measure of patient safety to help determine whether hospitals are penalized under two programs. One of those programs, the Hospital-Acquired Conditions Program, in December reduced Medicare reimbursements by 1 percent for 721 hospitals for their rates of preventable harms, such as serious blood clots, pressure ulcers, and accidental punctures and lacerations.
Serves them right, you might think. These hospitals are unnecessarily harming patients. That might be true if the test of their patient safety performance was scientifically sound. However, these programs have a serious methodological flaw: Many of their component measures are not based on reviews of the clinical record, but are rather derived from billing information, which produces a high rate of false positives. Indeed, for some of these measures, more than half of the incidents identified as preventable harm turn out to be false, once we review the clinical documentation. There can be many reasons for this. For instance, a patient may have had a pressure ulcer before admission that was not documented. Or a clot in a small vein might be mistakenly coded as a more serious clot known as a deep vein thrombosis. Continue reading “Health Care Shouldn’t Judge Itself by Flawed Tests” »