Efforts Under Way to Improve Assessment of Operative Competency

At the 2007 annual meeting of the American Academy of Otolaryngology Head and Neck Surgery (AAO-HNS), investigators from Johns Hopkins University and the University of Kansas reported on a needs assessment that identified a number of issues related to improving the formal assessment of operative competency among otolaryngologists-head and neck surgeons. These issues included (1) the difficulty of defining surgical competency, (2) recognition that the number of surgical cases performed may not determine surgical competency in all residents, (3) the potential problem that residents who require remediation are often identified only late in their residency, and (4) the dilemma that program directors and department chairs want to certify surgical competency in their residents, but they fear legal and other ramifications if they do not sign off on those residents who are surgically challenged.

This needs assessment was in response to the mandate by the Accreditation Council for Graduate Medical Education (ACGME) that now requires all graduate medical education specialties to incorporate into training programs demonstrated competency in six major core areas. One of these core competencies is patient care, which includes surgical competency. The mandate was generated, in part, by the need to show demonstrated surgical competency to the public, health care payers, and accrediting bodies such as medical licensure boards.

Ultimately, a number of centers will need to gain experience with the use of tools to test their reliability and validity, and the American Academy of Otolaryngology-Head and Neck Surgery as well as the American Board of Otolaryngology will have to indicate their agreement with this idea before otolaryngology programs will want to implement them widely.

-Robert Weatherly, MD

At the 2008 annual AAO-HNS meeting, investigators reported on what has been done over the past year to develop and implement assessment tools for operative competency in otolaryngology trainees.

Development of Assessment Tools

According to Nasir I. Bhatti, MD, of the Department of Otolaryngology-Head and Neck Surgery at Johns Hopkins University School of Medicine in Baltimore, the main task for improving assessment of operative competency is to develop assessment tools that are practical and feasible.

Although he said that attempts in the past have been made at developing more objective measurements of competency, such as examining electromagnetic waves from surgeons’ hands or assessing surgeons’ eye movements, these types of measurements are not very applicable to resident programs or to the everyday life of a surgeon.

click for large version

Figure 1. These 12 core procedures are necessary for attaining competence as a primary surgeon. Assessment tools have been developed for each core procedure.

Therefore, Dr. Bhatti and his colleagues at Johns Hopkins are collaborating with investigators from Canada and the United Kingdom to come up with objective measurements that are both feasible and can be generalized to residency programs worldwide.