The scientific literature is filled with studies showing that endoscopic sinus surgery (ESS) can be a highly effective therapy for patients with refractory chronic rhinosinusitis (CRS). Quality of life (QoL) scores, changes in endoscopic grading, and a host of other clinical endpoints all improve significantly (Int Forum Allergy Rhinol. 2014;4:823-827).
What is not so well documented, however, is whether those clinical improvements are “worth” the substantially higher up-front costs of surgery when compared with continuing medical therapy. That evidence gap, several rhinology experts argue, has led to persistent problems with payment denials from both government and private payers.
Those obstacles may soon be taken down a few notches, now that the first economic evaluation of ESS using sophisticated statistical modeling has shown that the procedure is in fact the most cost-effective treatment when long-term follow-up is factored into the equation (Laryngoscope. Published online ahead of print September 3, 2014. doi: 10.1002/lary.24916).
The economic model, applied to ESS by lead author Luke Rudmik, MD, MSc, as part of his masters thesis project for the London School of Economics, breaks new ground, according to coauthor Timothy L. Smith, MD, MPH, the director of the Oregon Sinus Center at Oregon Health and Science University in Portland and a member of the ENTtoday editorial advisory board. “This really is the holy grail of evidence for showing that a given therapy not only can improve clinical outcomes but does so in a cost-effective manner relative to other treatment options,” said Dr. Smith.
The economic model included two primary groups for comparison: patients with refractory CRS who underwent ESS followed by post-operative medication therapy, and those treated with medication therapy alone. The primary outcome was the cost per quality-adjusted life year (QALY). Several related outcomes were based in part on those QALY calculations, including incremental cost-effectiveness ratios (ICERs) and other tools for financially assessing the clinical interventions.
Because one of the main goals of the study was to determine whether the higher up-front costs of ESS—approximately $7,500—are justified, the investigators used a 30-year “time horizon” for follow-up. Although the study was a simulation model, the investigators stressed that the variables built into it, such as the probability and cost of peri-operative complications, outcomes after ESS, and the varying “health states” of patients, were based on real-world clinical outcomes from 168 patients who underwent ESS in studies funded by the National Institutes of Health (NIH) (Otolaryngol Head Neck Surg. 2010;142:55-63).
“Those studies yielded some of the most robust clinical data on endoscopic sinus surgery in the last decade,” said Dr. Smith, who participated in the NIH trials. “So we’re confident that they gave us a strong foundation for extrapolating cost effectiveness—and this is key—over a long enough time horizon to really give policymakers and payers something to think about.”
Looking at a reference case of refractory CRS, the investigators found that the sinus surgery strategy cost $48,838.38 and produced 20.50 QALYs. The medical therapy alone strategy, in contrast, cost $28,948.98 and produced 17.13 QALYs.
But those figures don’t tell the whole story, according to Dr. Rudmik, who is an assistant professor in the division of otolaryngology–head and neck surgery at the University of Calgary in Alberta, Canada. “You have to evaluate the ICER calculation to fully appreciate the cost effectiveness.” The ICER, he explained, “gives you the additional cost in relation to the additional benefit of the interventions you’re evaluating in an economic model.”
In this case, he noted, the ICER for the ESS strategy versus medical therapy alone strategy was $5,901 per QALY for the reference case. Although that number understandably increased when more complicated cases were factored in, even then, “ESS remained the most cost-effective decision, with a maximum ICER of $11,030 per QALY.” At this point, clinicians who don’t have a Master of Science degree in health economics may be thinking, “Are we done yet?” But Dr. Rudmik stressed that one more important statistical tool needs to be understood in order to grasp the significance of his cost-effectiveness model—it’s known as a probabilistic sensitivity analysis, a calculation “that gives you a percent certainty that an intervention is cost effective below a certain willingness to pay [WTP] threshold,” he said. “That is, what dollar expenditure is a government or private payer comfortable with, when it comes to paying for an intervention based on the expected long-term benefits?”
What is considered an acceptable WTP threshold? “We focused on a threshold of $50,000 per QALY, because most policymakers in the U.S. will accept a medical intervention that is below that threshold,” Dr. Rudmik said. And, here again, the study results favored the ESS strategy. The investigators found that there is a 74.5% and 77.8% certainty that sinus surgery is the most cost-effective strategy for managing refractory CRS, at a WTP threshold of $25,000 and $50,000 per QALY, respectively. Importantly, the “time horizon” analysis done in the study suggested that these cost-effectiveness thresholds began to kick in at three years of follow-up.
“Think of it this way,” Dr. Rudmik said. “If you’re a betting person as a policy maker or payer, the odds are around 75% that you’d be correct in supporting ESS because of its long-term clinical and cost effectiveness, whether that be via favorable reimbursement rates, coverage determinations, treatment guidelines, etc.”
David W. Kennedy, MD, a rhinology professor at the University of Pennsylvania in Philadelphia, is uniquely qualified to evaluate any long-term look at the merits of surgery versus medical therapy for refractory CRS. Dr. Kennedy is one of the pioneers in bringing modern methods for performing ESS to the United States and, in the 1990s, authored some of the first studies to measure the utility of ESS over an extended time period—nearly a decade in two of his studies (Laryngoscope. 1992;102(12 Pt 2 Suppl 57):1-18; Laryngoscope. 1998;108:151-157).
“But I have to tell you, I’m no healthcare economist and therefore cannot comment on the economic modeling aspect of the study,” Dr. Kennedy said. “So if I were asked to be a reviewer for this paper, my first question—and it is absolutely critical to the validity of its findings—would be a clinical one: That is, what was the severity of disease for these patients? Did they really have refractory disease?”
“My understanding from a recent conversation with the investigators is that the mean Lund-Mackay CT [computed tomography] scores for the patients in their database is 12, which equates to moderately severe disease,” Dr. Kennedy said. “That tells me that their patient selection criteria, at least as it pertains to disease severity, was sound.” (Information on Lund-Mackay scores was not included in the published study.)
A Few Too Many Assumptions?
Martin Citardi, MD, professor and chair of the department of otorhinolaryngology–head and neck surgery at the University of Texas Medical School in Houston, was a bit more willing to take
on the statistical modeling strategies used in the Rudmik study. He said the central question of cost effectiveness that the authors posed “is a good one to be asking” in this era of cost containment. “And I suspect that their conclusion is probably right. I am concerned, however, that their analysis is based on assumptions on top of assumptions on top of assumptions, so the foundation of their analysis may have some flaws.”
Dr. Rudmik agreed that any statistical model is only as sound as the quality of the variables built into it. “That’s why we took such great pains to scour the NIH studies for real-world patient data to make those assumptions solid,” he said. Moreover, to ensure that the economic model accounted for those clinical and financial variables as much as possible, “we didn’t just include the mean values for all of our data,” he stressed. “We also used the values in between the 95% confidence interval [CI] ranges for each parameter in the study.” In fact, “we must have run our sensitivity analysis at least 15,000 times, picking random samples for each parameter in the 95% CI range. So I do feel strongly that the model appropriately accounted for the inherent presence of uncertainty around the true value of each variable included in the model.”
Dr. Citardi pointed to another drawback to the study: its treatment of refractory sinus disease as a single disease entity. “That’s likely not truly reflective of what we see in clinical practice,” he said. “There are probably subgroups of patients with different disease profiles in whom sinus surgery is extremely valuable and cost effective, and there are others where it is likely to be of dubious benefit.”
Both Drs. Smith and Rudmik agreed that was a limitation to the study and actually identified it as such in their paper. They also cited another factor that makes it difficult to aggressively extrapolate the findings to clinical practice: The NIH patient data it relies on was not derived from randomized controlled trials comparing ESS and medical therapy. “Basically, there are no such trials,” Dr. Rudmik said. The reasons for that evidence gap are many, including the ethical problem of withholding surgery from refractory patients when it is clear that surgery may be superior to drug therapy alone. Plus, said Dr. Smith, these patients would need to be followed for the rest of their lives. “There’s no funding mechanism for that,” he added. Hence the need, at least initially, for an economic modeling approach.
Is It Really All about the Money?
And then there’s the larger question of whether economic considerations should be given so much prominence when weighing the relative benefits of treatment strategies for refractory sinus disease—or any otolaryngic procedure, for that matter.
Dr. Citardi noted, “There clearly are problems with the entire reimbursement paradigm for what we do as ENT physicians and surgeons. But it certainly is not unique to sinus surgery; it’s across the board.”
“Physicians sometimes get too caught up in the question of finances for a given treatment,” Dr. Citardi said. “But the payment challenge is separate from the issue of determining who is a good candidate for sinus surgery. In fact, I don’t ask patients about their ability to pay, or their insurance status, etc. Instead, I try to make the right decision based on the clinical facts at hand.”
The issue that is relevant to sinus surgery, he stressed, is the lack of any clinical trials identifying who is the best candidate for sinus surgery. “This is very ill-defined,” Dr. Citardi said. “And that’s on us, on otolaryngology, for not answering that question. If we can ultimately define the patient population for whom ESS is most beneficial, then we could go to governmental and private payers and make a strong case for more rational, consistent payment policies.”
The lack of such clinical data is exactly why Dr. Rudmik embarked on the economic model described in The Laryngoscope study. “We don’t yet have the definitive head-to-head trial to make these determinations,” he said. “But we do have our paper, and I think the data we cite is a good start.”
Are the findings good enough to serve as ammunition when fighting an ESS payment denial by a government or private payer? “It’s worth a try,” Dr. Smith said. “But remember, payers tend to take a short-term view of cost because of patient migration off their plans. So that initial $7,500 higher cost for sinus surgery, unfortunately, is what resonates most with many insurers—not any downstream benefits.”
For Dr. Rudmik, that type of short-term outlook is unfortunate. “I would challenge payers to find ways to serve the patient rather than ways to maximize profit. If they figured out ways to make their patients healthier and happier under their plans—such as paying for sinus surgery, which we’ve shown yields favorable long-term cost effectiveness—then maybe they wouldn’t be so likely to shop around for different coverage. Payers may well be surprised at how much patient retention—and savings—they could achieve.”
David Bronstein is a medical writer based in New Jersey.