NAPLES, Fla.-New mandated requirements for residents at hospitals require the young specialists to do a lot more than just attend classes; they have to be able to demonstrate that they have achieved some proficiency in specific areas of medicine.
However, the new requirements go well beyond just making sure residents are able to demonstrate proficiency. Their mentors and administrators have to be able to rate how well the residents are doing, said Brian Jewett, MD, Assistant Professor of Facial Plastic and Reconstructive Surgery at the University of Miami, Fl.
Survey Highlights Knowledge Gap
Furthermore a recent survey indicates that residents, faculty, and program directors are all uneasy about how well they can evaluate the skills that residents should be obtaining, Dr. Jewett said at the Triological Society’s Southern Section Meeting here.
He said that he and his colleagues sought to evaluate the degree of implementation of these requirements in otolaryngology residency programs. All program directors were surveyed to determine their methods for assessing residents’ attainment of competency-based learning objectives.
The Web-based survey was sent to 102 program directors; 39 of those directors responded. Dr. Jewett then performed an analysis to determine faculty and resident knowledge about assessing these competencies. The results he shared with the Triological Society indicated that there is room for improvement. He found:
- About 77% of residents and 33% faculty were not aware of the six general competencies;
- None of the residents or faculty could list all six general competencies; however
- The majority could identify some or all of the competencies from a list.
Those six general competencies, include:
Patient care: that residents are making informed decisions about diagnostic and therapeutic interventions, that they are developing and carrying out patient management plans, and that they are capable of performing procedures competently.
Medical knowledge: that residents develop analytical thinking regarding treatment issues and that they can apply basic and clinically supportive sciences to basic care.
Professionalism: that residents are familiar with the ethical principles pertaining to provision or withholding of clinical care, confidentiality of patient information, issues of informed consent, and business practices, and can demonstrate respect, compassion, and integrity.
Communication skills: that residents are able to give effective information exchange; will develop effective listening skills with the patient; will develop effective nonverbal, explanatory, questioning, and writing skills.
System-based practice: that residents understand the context in which they practice medicine, including how to partner with health care managers and service providers, and understand their social accountability by being cost conscious and by being aware of community resources.
Problem-based learning: that residents learn from experience and errors, show evidence of improved patient care, incorporate information technology into their practice, and learn to adapt to changing scenarios.
In addition, Dr. Jewett’s surveys found:
- At least 66% of faculty and 85% of residents did not feel comfortable explaining, implementing, or measuring the system-based practice competency; and
- At least 77% of residents and 50% of faculty did not feel comfortable measuring the remaining general competencies.
We have identified a significant knowledge gap in the understanding of these competencies as well as several challenges that remain in the implementation of validated measurement tools, Dr. Jewett said.
Why Change Assessment?
The issue grew out of attempts to standardize graduate medical education, he said-attempts that date to the earliest years of the Triological Society, with discussions on how to assess medical education found in minutes of meetings occurring as early as 1912.
The Accreditation Council of Graduate Medical Education (ACGME) outcomes project aims at assessing learning objectives based on competencies, developing valid and reliable methods of assessment, and developing outcomes data that facilitate continuous improvement.
Dr. Jewett said there are differences between the current accreditation process and what was sought previously. The previous model for accreditation focused on the potential of the program and the assessment of the structure and processes of the program, he explained. Typically, administrators were asked: Does the graduate medical education program comply with requirements? Has the program established objectives and a curriculum? Does it evaluate residents as well as the program itself?
New Focus for Accreditation
Today there is a new model for accreditation. That new model focuses on the outcomes of the program-what has been accomplished by the resident training program-with an emphasis on methods used to assess those accomplishments, Dr. Jewett said.
Instead of wanting to know if a resident has attended the necessary classes, the new accreditation program wants institutions to figure out a way of telling if that resident actually learned anything by going to the class, he said.
The typical new questions that are being asked are: Is there evidence that the residents achieved the learning objectives? What kinds of measurement tools were used to make those assessments? How is the institution going to demonstrate continuous improvement in its educational processes? he said.
We have identified a significant knowledge gap in the understanding of these competencies as well as several challenges that remain in the implementation of validated measurement tools. – -Brian Jewett, MD
Dr. Jewett said the ACGME outcomes project is expected to involve medical educators/physicians, residents, government and industry, academic health centers, private foundations, and non-physician health care providers. The outcomes project was part of a US Department of Education mandate that was promulgated in the 1980s, he continued. The project also was an outgrowth of demands from policy makers who were concerned about patient safety and privacy as well as about proper expenditure of health care resources, he said.
Complicating matters during the same period, hospitals were pressured to lower work hours for residents. That was supposed to give the residents more rest time to avoid resident-burnout and medical errors that were related to tired residents. However, the reduction in hours also resulted in a reduction in the time that training opportunities were available, Dr. Jewett said.
Change on the Wind
Session moderator Robert H. Miller, MD, Executive Director of the American Board of Otolaryngology in Houston, Tex., and Chair of ENToday’s Editorial Board, said the new criteria for evaluating residents is designed to get us beyond what we have been doing for the past 100 years.
Co-moderator, Paul Lambert, MD, Chairman of the Department of Otolaryngology-Head and Neck Surgery at the Medical University of South Carolina in Charleston, said one goal of the new criteria is to assure that residents develop procedural competencies.
Dr. Jewett noted that the vast majority of the individuals he surveyed know that the winds of change are blowing. About 85% of them anticipate changes in methods of residents’ assessment, he said. But the bottom line aim, he suggested, is to stimulate discussion so we can optimize outcomes in graduate medical educations.
©2006 The Triological Society