For a moment, consider that you work at a primary care clinic, and your team needs to improve performance on annual foot exams on patients with diabetes — a critical step to prevent foot ulcers and amputations.
At your monthly meeting, staff members enthusiastically suggest solutions, such as using robocalls to urge patients to schedule appointments when they need exams. Everyone is ready to get started.
So what happens next? This is where so many patient safety and quality improvement projects skimp on a critical phase: They start making changes before identifying all factors that will influence their success or creating an evaluation plan that fully considers these factors.
Care teams often come up with innovative ideas to prevent local harms, only to realize after some encouraging results that their intervention is something that might be spread beyond their walls. Yet too often, weak study designs can make it hard to know why it worked, if it worked at all. The flaws can come in a number of forms, such as neglecting to consider and measure key variables, introducing too many interventions at once, and failing to collect sufficient data.
To avoid these pitfalls, it's critical to think about project design and measurement from the beginning, rather than as an afterthought. Here are several tips for getting started right.
Find a Framework
As a first step, identify a conceptual framework that describes the system you are trying to improve and that helps you consider all of the factors that might influence your success.
If you want providers to follow evidence-based practices, such as performing annual foot exams, you might select a behavior change framework, such as the Interdisciplinary Conceptual Framework for Provider Behavior Change. Another commonly used framework is the Consolidated Framework for Implementation Research, which provides categories of important influences that affect a wide range of improvement efforts. Donabedian's Structure, Process, Outcomes model is another straightforward framework.
Inherent in all improvement work is a theory of what actions will produce the desired change: If we do X, then Y will occur. Frontline staff members may not think of themselves as qualified to develop theory. But as authors of a BMJ Quality and Safety article pointed out early this year, there is always an implicit explanation for what will produce change. Unfortunately, those explanations often go unexamined in local improvement projects. By following a conceptual framework and developing a theory, teams can make their reasoning explicit, examine their assumptions and weigh the causal logic of proposed interventions, they write. (See their article for examples of theories developed for actual improvement projects.)
Create Your Improvement Map
A conceptual framework can guide your clinic team in creating a map of the process you are trying to improve. Once you select your goal — improving performance on foot exams — trace all steps required to make that happen, as well as the contextual factors that could influence outcomes. You may find that you can simply "plug in" answers on the framework. Often, the framework will serve only as a guide, and your improvement map will take a different shape.
As you create the map, consider the pre-existing characteristics of the system you seek to change. You might take stock of the number of physicians on staff, availability of staff to help with diabetes care, attitudes toward quality in the clinic and whether the clinic rewards high volume more than higher quality care. Review the systems in place that might make this task easier or more difficult. For example, does the clinic have a registry of patients with diabetes? How hard is it to get information about them?
Now, think about clinicians. What factors, such as busy schedules, attitudes or training deficits, might prevent them from conducting foot exams as often as needed? Do they know how many of their patients are overdue for foot exams?
Next, think about characteristics of the desired behavior change. How much time does it take to complete a foot exam? Is the idea consistent with what providers already believe is important for patients with diabetes? Are there any available, simple solutions?
Think about patients. Consider the factors that help get patients from their living rooms into the clinic. How will you get them to schedule an appointment? Are there any barriers — financial, travel or otherwise — for them to visit the clinic?
Once this map is in place, it will help you to understand what your barriers are, where your process is breaking down and what may help you achieve the results you are seeking.
Select Your Interventions and Isolate Your Variables
Before you greenlight a given intervention, be sure it addresses the pre-existing conditions. For simplicity's sake, let's assume that your clinic identifies two broad barriers to improvement: Patients with diabetes aren't setting up their annual appointments, and time-pressured physicians sometimes forget to conduct the exam.
To address these problems, the clinic decides to roll out three interventions:
- Robocalls urging those who are due for an exam to schedule appointments, followed by reminder calls
- Medical assistants removing the shoes and socks of patients with diabetes in the exam room, as a reminder to physicians (Colleen Leavitt, M.D., of East Baltimore Medical Center, gets credit for suggesting this approach in a recent course.)
- Provider-level dashboards showing the percentage of eligible patients who are up to date on their foot exams
How will you know which of these three interventions is responsible for any improvements? If you start all three at once on all of your patients, you will be hard-pressed to know.
Here, you generally have two approaches. One is to phase in interventions over different time periods so you can measure the incremental improvements (hopefully) after each step. Another approach involves rolling out different interventions on separate patient populations. This might not be practical in a single clinic, but it could work in a project spanning several clinics.
For best design, you can designate a control group of patients receiving no interventions to separate out what likely would have happened if you had not intervened. There may be other things happening outside of your intervention that could affect patients’ results.
Measure Everything You Can
You need to do more than measure your main outcomes — in this case, the foot exam rate. It's important to test the success of different interventions in improving processes. For instance, have robocalls helped to reduce the no-show rate for patients with diabetes? That is a critical step toward improving the foot exam rate, but it is probably not sufficient by itself.
You need to think about measuring how well the interventions have been implemented as well. Maybe the medical assistants are telling patients to remove shoes and socks, but patients get cold and put them back on. Patients might need more explanation about why they are removing their footwear, and medical assistants might need to check that shoes are still off when the clinician arrives.
Remember to try to gather data from different sources that represent different perspectives on the project. Together they can help tell the whole story of the effort. What if your efforts improve the patient outcome but engender ill will among physicians or nurses? Surveys can help you assess this and decide if the improvement is worth the hit to morale. How do foot exams affect productivity? If these exams cause schedules to back up, your solutions may not be sustainable without more staffing.
Get Assistance
Maybe you're asking: Is this something that frontline clinicians and staff members can do? It's a reasonable question. Quality and patient safety improvements are team efforts, involving frontline staff members, leaders, quality and safety specialists, and others. But clinicians and other frontline staff members know their patients, workflows and barriers to improvement better than anyone else. The people who are being asked to change need to be involved in the process of creating the improvement map and suggesting solutions.
Teams often need help with measurement. You can work with your hospital quality improvement department to identify available measures, and information technology specialists can help you to obtain additional data you need. You may need help from a staff member, such as a clinical research nurse, to collect data, refine your study design and analyze your results.
Even if you don't plan to publish your results in an academic journal or seek a grant to spread your success across many hospitals or clinics, developing your conceptual framework and measurement plan from the start can help you to avoid second-guessing. And if you do choose to share your success with the world — or the unit down the hall — you will have a better sense of what worked and why.
