Measuring what matters in hospitals
Hallway health care – gurneys with sick patients lining hospital corridors – is one of the most graphic representations of the pressures on Ontario’s health system today. We know that the equivalent of more than 10 large, 400-bed hospitals are filled to capacity each day by patients who don’t need the level of services hospitals are designed to provide. They are waiting for more appropriate placement in long-term care, rehabilitation, home care and assisted living.
Tackling these and other system challenges requires an accurate assessment of their scope and scale. Every health care system needs to measure how it is doing so it can improve and so Ontarians know whether the system is moving in the right direction and if they are getting good value for their money.
To properly support the priorities for the system, it is important that what we measure helps us identify and focus our efforts on the most pressing concerns. It is also important not to overwhelm health care professionals with a burdensome requirement to measure too much. As health quality guru Dr. Don Berwick wrote in 2016, excessive and mandatory measurement “is as unwise and irresponsible as is intemperate health care”. Measurement should provide meaningful information on performance, so providers can begin the quality improvement process, if required.
The Ontario Hospital Association and Health Quality Ontario are working together to stop excessive measurement and prioritize what matters. The objective is to ensure that investment and efforts to measure and report performance indicators in hospitals support better patient outcomes and are aligned with Ontario’s health system strategy.
Depending on the size and type of institution, senior leaders in hospitals in Ontario have identified more than 300 indicators that cross their desks from different organizations, in different report formats, on different time cycles and sometimes with overlapping indicators using different definitions. This is in addition to what a hospital may choose to collect on their own to manage and improve care and operations. Some of these quality improvement indicators are duplicative or not aligned with one another. Additionally, hospitals are asked to individually calculate and submit indicators when they might be more efficiently monitored centrally.
When we asked hospital leaders what was useful in all this reporting for managing and improving quality, the number of indicators shrunk by about a third. This didn’t mean that there weren’t gaps (e.g. what truly matters to patients), but it did highlight that there was a lot of measurement and reporting happening that was clearly of low value.
One of the key issues we identified is that our system tends to add new measures but is reluctant to retire measures even when they are not actively being used to make decisions or improvements. We don’t have clear criteria for when a measure should be modified or retired. And even when we add measures, we don’t always have a clear plan for how that data will be used. In the worst cases, there is data being collected routinely and no one is using it or looking at it.
Working closely with senior hospital and system leaders we have committed to:
1. Significantly reduce what we currently measure in hospitals, retiring measures of little to no value;
2. Revisit what we measure on a routine basis (i.e. annually) to make sure it continues to be of value;
3. Exercise much more discipline when we add measures about the plan for how it will be used;
4. Distinguish between what is important enough to be publicly reported vs. what is a lower priority but should still be monitored vs. what is low value and should be retired.
5. Recommending a centralized system where indicator performance is monitored and communicated back to providers or stakeholders when an issue has been identified.
An emphasis on a smaller number of system measures necessarily requires more discipline. It forces us to make important choices at the individual hospital and system levels, and it acknowledges that improvement can happen with the right focus. The hard work of quality improvement is not measurement. It’s the incremental and iterative steps frontline health care professionals and teams take to make patient care better. Measurement is vital to knowing whether these improvements make a difference. But excessive measurement can be a needless distraction from the hard work of improvement.
Measurement has always been a critical part of the delivery of quality care and as Ontario embarks on a restructuring of care delivery, it is clear the province’s hospitals need to be well-aligned to measure what matters.
Anna Greenberg is Interim President and CEO, Health Quality Ontario and Anthony Dale is President and CEO, Ontario Hospital Association.