Developing Quality Measures in Otolaryngology-Head and Neck Surgery

Overall, Dr. Amin emphasizes the need for otolaryngologists to determine which outcomes and measures are meaningful and worth measuring, and not leave it to non-otolaryngologists, who may develop metrics that are flawed or even meaningless.

By developing our own metrics, we have the opportunity to review them and modify them until we are satisfied that they are meaningful, she said. We do not expect the metric to be perfect from the start.

Consider the Stakes

Another way to look at this balance of metrics that measure both the larger outcomes of a practice as well as the more narrow outcomes is to consider what is at stake.

For David Roberson, MD, of the Department of Otolaryngology and Director of Implementation, Communication, and Education in Patient Safety and Quality at Children’s Hospital, Boston, there are two types of measurements that physicians can use based on the consequences that will result from the outcome.

For high-stakes outcomes, such as payment or renewal of licensure, performance measurements should be based on high-quality prospective studies that use carefully defined measures to determine statistically significant outcomes.

But for lower-stakes outcomes, such as trying to improve follow-up time on lab results in a practice, using less robust or quick and dirty measurements is sufficient.

For physicians who are concerned about quality and performance measures, determining which type of measurement to use is important.

The important thing is that there is not only one type of measurement, said Dr. Roberson. When you’re trying to make things better, you should use quick and cheap measurements and repeat them frequently. If you have something high-stakes, such as your licensure or your payment, you should reject that concept and insist on high-quality measurements.

According to Dr. Roberson, physicians are well versed and educated in turning to the medical literature to gather information on measurements based on prospective data from randomized clinical trials, but are not schooled in the other type of measurement that is commonly used to improve quality in other industries.

Called rapid-cycle improvement, this quick and dirty way of measuring an outcome is used when trying to make something better in one’s practice. This type of measurement is useful in everyday practice for physicians who want to improve the quality of care within their practice but without the expense, time, and resources needed to conduct a randomized controlled trial.

The problem is that physicians are not familiar with this idea of ‘quick and dirty’ measurement, said Dr. Roberson. But it is common sense that if you want to make something better in your practice, use something that doesn’t take too much time or energy.

We have to take a leadership role in defining and improving quality for our specialty. If we don’t do it, someone else with an ulterior motive will do that for us. So if this is really going to be about patients and making patients better, we have to step up and take a leadership role.
-David R. Nielsen, MD

For Dr. Roberson, the aim is to help physicians get involved with quality improvement in a way that makes sense to them. Many of the improvements in patient care can be done by tweaking systems regularly-say, every couple of weeks or months. You try to make a lot of small changes and evaluate quickly whether they work or not, he said.