How Crowdsourcing Is Maximizing the Impact of Medical Research

While crowdsourcing for the general population might mean getting on Facebook and asking friends to recommend a plumber, reading reviews on Yelp, or contributing information to a Wikipedia page, medical researchers are now using crowdsourcing to save research time, expedite diagnoses, and pursue groundbreaking discoveries powered by new technologies.

In 2014, The Michael J. Fox Foundation for Parkinson’s Research provided smart watches to study participants with Parkinson’s disease to track their movements and the management of their illness. In 2015, The American Heart Association (AHA) and the Patient-Centered Outcomes Research Institute (PCORI) released a new best-practices guide that endorsed crowdsourcing as a way to “… accelerate medical research and improve global cardiovascular health,” according to an AHA news release. The guide, Crowdsourcing Medical Research Priorities: A Guide for Funding Agencies, is the work of PRANCCER (Precision Medicine Advances using Nationally Crowdsourced Comparative Effectiveness Research) and a joint initiative of the AHA and PCORI.

And, last year, Apple released ResearchKit, an open-source software that runs on the iPhone and can make use of the phone’s sensors to collect data such as daily step counts and heart rates for research purposes. The software can also be used for facial recognition for autism diagnosis and to track seizures, among other things. Medical researchers can use ResearchKit to develop a variety of apps that create tools such as consent forms, real-time tasks, and surveys with modules that can be customized and shared. The software also works with wearables like the Apple Watch, which, like other smart watches, can monitor users throughout the day. For airway disorders, for example, otolaryngologists might employ wearable technology to track study participants’ exercise tolerance or how much they sleep.

Beyond collecting patient data, physicians and researchers are using the power of the crowd to gather expert opinions and delegate microtasks to large groups of people, among other things. Karthik Balakrishnan, MD, MPH, an otolaryngologist and pediatric airway disorders specialist at the Mayo Clinic in Rochester, Minn., has used crowdsourcing to consult with peers, patients, and patient’s parents, delivering surveys on their experiences at the clinic or the hospital and using that data to reach a consensus and improve the quality of both visits and treatment.

Dr. Balakrishnan first became interested in crowdsourcing when he heard innovator and patient empowerment expert Lucien Engelen speak at a conference about crowdsourcing the locations of automated external defibrillators (AED) throughout the Netherlands to help save lives during cardiac emergencies. Currently, Dr. Balakrishnan is working on a grant-funded project for which he recruits a carefully selected population of patients’ parents to join a private Facebook group. “We’ve selected the type of patient where the parents tend to be educated about the condition and very invested in the child’s care,” he said. He added that the purpose of this project is two-fold: The first goal is proof of concept, to show he can get useful results from crowdsourcing in this way; the second is the increased contact with parents whose children’s disease is his specialty.

The Delphi Method Steps into the 21st Century

Dr. Balakrishnan also conducts studies using the Delphi Method, a research process developed in the 1950s to predict how technology might impact war. The method uses groups of experts to anonymously answer surveys, gather statistics on the group response, and reduce the range of responses to arrive at a consensus when searching for answers to complex issues.

An example of the use of this method is a situation in which there was no data to make any kind of conclusion or recommendation. In real time, during the Cold War, researchers wanted to know the likelihood that Russia would launch a missile attack and determine how experts would predict this. “It’s hard to even think about what factors you would put together to reach some sort of conclusion,” said Dr. Balakrishnan, adding that the idea is that in the absence of data you can use expert opinion and expert experience. The beauty of the Delphi Method, he added, is that it corrects for the fact that if you ask 20 experts about something, you might get 20 different answers, and everyone thinks they are right. “And that’s certainly true in medicine,” he said. “We all like to think we’re right.”

The Delphi Method has been used worldwide for more than half a century, but advances in technology mean that, rather than gathering experts in a room at a conference and handing out paper surveys, it can now all be done online, making use of experts around the world. Dr. Balakrishnan used it in a study he conducted on pediatric laryngotracheal reconstruction (Laryngoscope. Published August 27, 2018 online ahead of print. doi: 10.1002/lary.27445), as well as in a Mayo Clinic-led study on pediatric aerodigestive programs he co-authored (Pediatrics. 2018;141(3). pii: e20171701), and he has several other similar studies in progress. He added that, while he has probably used the Delphi Method more than many otolaryngology investigators, it is gaining in popularity among researchers in the specialty.

Erynne A. Faucett, MD, a pediatric otolaryngologist for the division of otolaryngology at Phoenix Children’s Hospital in Arizona, said that she likes the fact that the Delphi Method, in contrast to other data gathering techniques, uses a multiple-iteration design to develop an agreement. “This feedback process may encourage participants to reassess their initial judgments about the information provided in previous versions,” she said, adding that it can prevent group pressure for conformity and the influence of dominant personalities, as well as allowing respondents to remain anonymous, if needed.

Dr. Faucett, whose research has been in educating medical students and residents, was introduced to crowdsourcing via the Delphi Method in fellowship when she and her colleagues were trying to figure out how to create an assessment tool for particular surgical procedures in pediatric otolaryngology. “We used the Delphi Method to bring experts in our field … to a consensus regarding creating assessment tools for particular surgical procedures,” she said.

She has also recently been involved in a project supported by the American Academy of Otolaryngology–Women in Otolaryngology Endowment grant, which involved a modified Delphi consensus study to identify high priority areas and associated gaps in current leadership and professional development resources for women in otolaryngology and surgery. “To increase our likelihood for successful identification of these priority areas, we utilized a Delphi consensus approach, which has been seen more in health care to achieve agreement/consensus among recognized experts on particular topics,” she said. With the topics identified via this method, Dr. Faucett and her colleagues will introduce a coaching curriculum to address these high priority needs and create an educational audio podcast, along with associated print materials.

Microtasking

Crowdsourcing isn’t just for tapping into the views and assessments of other experts. Sometimes researchers use crowdsourcing to delegate time-consuming tasks that are very small, also known as “microtasks,” to people who are not experts but do have basic skills that can help them earn a little money in their spare time.

Tammy Chang, MD, MPH, MS, a practicing family physician, an assistant professor at the University of Michigan Medical School in Ann Arbor, and a health policy researcher who has studied obesity, recently used the CrowdFlower (now called Figure Eight) crowdsourcing platform to carry out microtasks for her research analyzing the accuracy of websites that contained material about weight gain in pregnancy. “There are gold standard guidelines about how much you’re supposed to gain depending on how much you weigh right now,” she said.

For the project, Dr. Chang’s team needed people to look at the hundreds of websites that come up when someone Googles “weight gain and pregnancy” to see if those gold standards were reflected on each site. “For that kind of task,” she said, “you don’t need to be a physician or a researcher. You just need to be someone who can read and see if the guidelines that were published by the National Academy of Medicine were somewhere on each web page.” Through crowdsourcing, it took just days to review hundreds of sites.

One concern, Dr. Chang said, was knowing whether members of the crowd were performing the tasks accurately and doing a good job. However, the platform had an interesting setup that put her mind at ease: Those completing the microtasks received a bundle of work and were told there was a test site in it that Dr. Chang had already evaluated. If the microtaskers got the test site wrong, their work was automatically thrown out.

Privacy Concerns

The downside to crowdsourcing is the risk of a security breach, because most crowdsourcing uses an interface to transmit information, and there’s always the possibility that the platform could be hacked.

Dr. Balakrishnan noted the concern, for instance, that if a patient posts that they’re unhappy with their healthcare provider and that provider sees the post, this may affect the physician–patient relationship. “If we’re asking patients to share private information with us,” Dr. Balakrishnan said, “then we owe it to them to treat it with respect.” Part of that respect at large institutions means putting the project before the Institutional Review Board to screen for potential violations of privacy or poor data protection, he said. Another part is making parents aware of how important it is to think hard before posting about their child in an online group and ensuring that their post doesn’t have anything specifically identifiable about that child.

Douglas R. Sidell, MD, a pediatric otolaryngologist and assistant professor of otolaryngology/head and neck surgery at Stanford University in California, also underlined the importance of patient respect. “When patients are involved, we must consider not only patient privacy, but we also need to be respectful of patients on all fronts, regardless of whether or not we are operating within the confines of legality,” he said. “This becomes particularly true when using social media resources. We need to maintain secure information platforms across all points of contact with the information source, and we need to maintain de-identified patient information whenever possible.”

Dr. Sidell is currently working with colleagues from the Mayo Clinic, Johns Hopkins University, and Stanford University to identify specialists around the world who are using a novel chemotherapy for the treatment of respiratory papillomatosis. “We are using this process in an attempt to achieve consensus pertaining to the use of this medication,” he said. He has employed various anonymous response systems such as Qualtrics and other automated survey platforms and said he cautiously enlists participation via Facebook and uses WhatsApp groups to communicate with physicians in different parts of the world. “I tend to crowdsource ideas rather than tangible commodities,” he added.

Playing Catch Up

Although crowdsourcing may seem new to some researchers, Dr. Chang said medicine is behind the curve when it comes to using this resource and that crowdsourcing is the bread and butter research of other areas, such as the social sciences. “It’s such a low risk and high reward thing to try,” she said, “I invite health services researchers to look into it and see if it might decrease their burden. It doesn’t cost very much but could be really useful.”

She also advises physicians who generally avoid using technology to try to keep an open mind and let their patients be their guides. “Your patients are using a lot of different technology. Be open to how they’re using it and why they’re using it. Most of the time it’s because it’s enhancing their lives in some way. We have to ask ourselves how we can leverage that to help us understand their lives better.”

---

Renée Bacher is a freelance medical writer based in Louisiana.