Developing Quality Measures in Otolaryngology-Head and Neck Surgery

A movement underfoot in industry is rapidly infiltrating all branches of medicine, and specialties, such as otolaryngology-head and neck surgery, are being encouraged to join the ranks. Quality improvement is not new to medicine, but the increased mandate by insurance companies, government agencies, and patients for physicians and health care systems to demonstrate quality care is placing new demands on specialties to develop metrics that accurately and adequately measure quality within their field.

This is probably one of the most crucial things that is going to affect all physicians within the next 10 years, said David R. Nielsen, MD, Executive Vice President and Chief Executive Officer of the American Academy of Otolaryngology-Head and Neck Surgery/Foundation (AAO-HNS/F).

For Dr. Nielsen, who spoke during a miniseminar on this topic during the recent annual meeting of the AAO-HNS in Chicago, specialties such as otolaryngology-head and neck surgery need to take a lead in developing their own measurements of quality to, above all else, keep the focus on the patient.

We have to take a leadership role in defining and improving quality for our specialty, he said. If we don’t do it, someone else with an ulterior motive will do that for us. So if this is really going to be about patients and making patients better, we have to step up and take a leadership role.

Impetus for Developing Quality Measures

Since the publications To Err is Human and Crossing the Quality Chasm in 2000 and 2001, respectively, by the Institute of Medicine (IOM), attention on and demand for improvements in the quality and measurement of outcomes in health care have increased. Pinpointing unacceptable high rates of morbidity and mortality, along with rapid increases in health care costs, these reports highlighted the need for the development and implementation of quality improvement measures in all branches of medicine.

The need to demonstrate quality in medical care is coming from many sectors-Medicare, insurance companies, employers, and the public-all of which have different criteria to satisfy. For payers, quality is linked to pay. For patients, quality is linked to feeling better. For government-sponsored programs such as Medicare, quality is linked to political will and distributing care with limited resources. For physicians, quality is linked to expected medical outcomes.

Everybody defines quality differently, and the biggest challenge we have is to make sure the focus stays on the patient, said Dr. Nielsen.

To this end, organizations such as AAO-HNS are looking at ways to develop metrics within their own specialties that reflect appropriate and accurate performance measures. Not only will this proactive approach help keep the focus on the patient, but it will also defend against external measures of quality being used to, for example, pay or not pay physicians based on purely administrative or financial performance.

For physicians who are not yet convinced that they need to address issues of quality and even participate in developing metrics in their own practices, the need to get on board is coming from stipulations already in place by organizations, such as health plans and Medicare, who are demanding demonstrations of quality for payment.

Further incentive is coming from the need for physicians to now demonstrate their participation in quality improvement to maintain board certification, as well as renewal of state licensure.

All these demands are pushing physicians into the quality movement. According to Dr. Nielsen, it is up to organizations like the AAO-HNS to respond to these demands and help provide measures for its members. We want to be at the forefront to make sure these demands are relevant, valid, patient-centered, and not just being made up by someone who wants to make a buck off the system, he said.

What the Academy is Doing

In an attempt to get otolaryngologists and head and neck surgeons involved in developing and using performance measures, the AAO-HNS is urging members to review and critique measures pertaining to their specialties utilized by a number of organizations (see sidebar, page 13).

The main help we ask of our members is to review measures being developed by other organizations that may affect the field of otolaryngology, said Jenissa Haidari, MPH, Senior Program Manager for Quality Improvement for the AAO-HNS.

From the AAO-HNS perspective, performance measures should be focused on the following:

• Improving quality of care and safety.
• Relevance to medical and health outcomes.
• Based on data that is fully risk-adjusted.
• Based on fair and accurate models for attributing care.
• Allowing physicians to review and correct performance data.

One program that the AAO-HNS has urged physicians to get involved in is the Physician Quality Reporting Initiative (PQRI). Initiated by the Centers for Medicare and Medicaid Services (CMS), this national program provides bonus pay for physicians who report using performance measures related to otolaryngology. (A complete list and details of PQRI performance measures related to otolaryngology can be found at www.entnet.org .)

According to Ms. Haidari, most of the measures from the progam that otolaryngologists are using involve perioperative care-including selection, timing, and discontinuation of antibiotic prophylaxis, as well as use of venous thromboembolism prophylaxis when indicated in all patients.

Despite a strong initial push to get members to participate in this program, however, reporting among otolaryngologists is quite low, said Ms. Haidari, primarily because many of the measures do not apply to the members or members do not meet the requirements for reporting (e.g., reporting requires a certain caseload).

According to Dr. Nielsen, the academy is also asking members to get involved in developing metrics. We need evidence-based guidelines that we can use as the foundation for building evidence-based performance measures that will serve as foundation for evidence-based pay for performance, he said.

Need to Develop Own Metrics

To ensure that appropriate and accurate measurements are developed, otolaryngologists-head and neck surgeons will need to play a more active role in developing their own metrics.

To do this, physicians will need to focus first of all on metrics that are useful. According to Manali Amin, MD, an otolaryngologist at Children’s Hospital, Boston, there are several marks of a useful metric: It is easy to measure; it encompasses many physicians; it measures a common clinical problem or procedure; it measures a procedure or treatment that has some associated morbidity, complication, or cost; and, perhaps most important, it serves as an impetus to change.

This last part is key, said Dr. Amin. A metric is useful only if after seeing the results a surgeon, for example, is compelled to make a change.

Of critical importance is the need to define a metric in an exact way so that others can also apply the metric to their practice. For example, Dr. Amin’s group has come up with a metric for measuring post-tube otorrhea rates. We have defined drainage from the ears between seven and 30 days after placement of tympanostomy tubes, she said. We are interested in looking at this metric to determine what our rate actually is and whether there are factors that we can adjust to reduce the incidence of otorrhea.

This is an example of what Dr. Amin defines as a metric to measure a narrowly defined outcome. Along with this type of metric, Dr. Amin emphasized the importance of practices to develop metrics that also measure the broad overview outcomes of a practice. In our practice, we have chosen to use our morbidity and mortality rates as a broad overview of individual outcomes, she said.

Overall, Dr. Amin emphasizes the need for otolaryngologists to determine which outcomes and measures are meaningful and worth measuring, and not leave it to non-otolaryngologists, who may develop metrics that are flawed or even meaningless.

By developing our own metrics, we have the opportunity to review them and modify them until we are satisfied that they are meaningful, she said. We do not expect the metric to be perfect from the start.

Consider the Stakes

Another way to look at this balance of metrics that measure both the larger outcomes of a practice as well as the more narrow outcomes is to consider what is at stake.

For David Roberson, MD, of the Department of Otolaryngology and Director of Implementation, Communication, and Education in Patient Safety and Quality at Children’s Hospital, Boston, there are two types of measurements that physicians can use based on the consequences that will result from the outcome.

For high-stakes outcomes, such as payment or renewal of licensure, performance measurements should be based on high-quality prospective studies that use carefully defined measures to determine statistically significant outcomes.

But for lower-stakes outcomes, such as trying to improve follow-up time on lab results in a practice, using less robust or quick and dirty measurements is sufficient.

For physicians who are concerned about quality and performance measures, determining which type of measurement to use is important.

The important thing is that there is not only one type of measurement, said Dr. Roberson. When you’re trying to make things better, you should use quick and cheap measurements and repeat them frequently. If you have something high-stakes, such as your licensure or your payment, you should reject that concept and insist on high-quality measurements.

According to Dr. Roberson, physicians are well versed and educated in turning to the medical literature to gather information on measurements based on prospective data from randomized clinical trials, but are not schooled in the other type of measurement that is commonly used to improve quality in other industries.

Called rapid-cycle improvement, this quick and dirty way of measuring an outcome is used when trying to make something better in one’s practice. This type of measurement is useful in everyday practice for physicians who want to improve the quality of care within their practice but without the expense, time, and resources needed to conduct a randomized controlled trial.

The problem is that physicians are not familiar with this idea of ‘quick and dirty’ measurement, said Dr. Roberson. But it is common sense that if you want to make something better in your practice, use something that doesn’t take too much time or energy.

For Dr. Roberson, the aim is to help physicians get involved with quality improvement in a way that makes sense to them. Many of the improvements in patient care can be done by tweaking systems regularly-say, every couple of weeks or months. You try to make a lot of small changes and evaluate quickly whether they work or not, he said.

However, he stressed that you do not want outcomes based on quick and dirty measurements to be used for determining things like pay or renewal of licensure.

We should let policy makers know that we’re happy with ‘quick and dirty’ measures if what’s on the table is simply a quality improvement effort, he said. On the other hand, if what’s being proposed is a high-stakes measure-something that will be used for payment or licensure-we should absolutely insist on high-quality, statistically meaningful measures.

Quality Relates to Both Physician and System Performance

Although performance measures often focus on individual physician performance, Dr. Roberson emphasized that physician performance, as well as the system in which the physician performs, are integral to assessing quality in modern medical practice.

It is not enough in 2008 to be a good doctor, he said. A surgeon in 2008 is both a surgeon and the leader of a high-performance team.

To this end, any measurement of quality needs to take into consideration the entire system in which the patient moves throughout his or her care. For Dr. Roberson, this highlights the ethical obligation physicians have to improve their own skills and how they interact with the system, as well as being involved in improving how the system works overall. For example, he said, if your resident and ICU nurse do not get along, put that on your radar screen.

Get Involved

With increased recognition of the high cost of medical care and the gaps in quality of care, efforts are under way to develop and implement performance measures to ensure delivery of the highest quality of care at the lowest cost. Over the next decade, physicians will increasingly be asked to demonstrate performance based on established metrics that will determine what they are paid and whether their licenses are renewed. To ensure that these metrics are actually meaningful and indicative of true quality that focuses on patient care, physicians need to get involved in developing relevant performance measures. Otolaryngologists-head and neck surgeons who need help navigating the different quality improvement initiatives or want to get involved in developing performance measures can contact the AAO-HNS at qualityimprovement@entnet.org.

A Tale of Two Organizations

American Academy of Otolaryngology-Head and Neck Surgery
Alexandria, Virginia
A membership organization involved in education, research, and advocacy for otolaryngologists.

American Board of Otolaryngology
Houston, Texas
The organization that certifies otolaryngologists and is responsible for the Maintenance of Certification program.

Medical Organizations Developing, Endorsing, or Utilizing Performance Measures

• Ambulatory Care Quality Alliance (AQA)
• Centers for Medicare and Medicaid Services (CMS)
• National Committee for Quality Assurance (NCQA)
• National Quality Forum (NQF)
• National Surgical Quality Improvement Program (NSQIP)
• Physician Consortium for Performance Improvement (PCPI)
• Surgical Quality Alliance (SQA)
• Specialty societies

©2008 The Triological Society