Researchers at UHN’s Krembil Research Institute and Princess Margaret Cancer Centre have identified the need for improved standards for survey-based research in the field of general surgery.
“Many people see surveys as a quick and easy approach to collecting data, but it is actually quite challenging to develop a high-quality survey that provides meaningful insights,” explains Dr. Fayez Quereshy, a co-author of the study and a Krembil Clinician Investigator. “Unfortunately, few guidelines exist to help researchers design surveys that are targeted to health care professionals.”
In survey research, scientists typically collect information about a population of interest using questionnaires. Survey studies allow researchers to obtain data quickly and at a low cost, but the conclusions drawn from surveys are only as strong as the quality of the survey methodology.
To explore the quality of survey research in the field of general surgery, the research team examined the methodology of more than 270 survey studies published in surgical journals over the past 18 years. The team was led by Dr. Sami Chadi, a Clinician Investigator at the Princess Margaret Cancer Centre, and research fellow and PhD candidate Dr. Keegan Guidolin.
The researchers examined several factors related to the overall quality of a survey’s methodology, including whether the authors used an existing questionnaire or developed their own, whether they followed specific steps when developing their questionnaire and how they reported survey details.
The study revealed several shortcomings in how surveys are carried out and described. For example, only 14% of studies used a previously developed and accepted questionnaire, and more than 20% of studies did not explain how the questionnaires were developed. Additionally, 63% of studies failed to report how the researchers managed incomplete questionnaires.
Based on their findings, Drs. Chadi and Quereshy recommend that researchers apply standardized methods when they conduct surveys. They encourage researchers to adapt existing questionnaires whenever possible and follow steps to assess questionnaire clarity and completeness when developing their own. They also encourage researchers to describe key survey details such as response rate and methods for managing missing data when they publish their results.
“We need to ensure that the results of surgery-related surveys are reliable,” says Dr. Chadi. “Survey studies often inform policy changes, so they can have far-reaching effects.”
This work was supported by The Princess Margaret Cancer Foundation and the UHN Foundation.
Guidolin K, Wexner SD, Jung F, Khan S, Deng SX, Kirubarajan A, Quereshy F, Chadi S. Strengths and weaknesses in the methodology of survey-based research in surgery: A call for standardization. Surgery. 2021 Feb 16. doi: 10.1016/j.surg.2021.01.006.
(L) Dr. Sami Chadi is a Clinician Investigator at the Princess Margaret Cancer Centre. (R) Dr. Fayez Quereshy is a Clinician Investigator at the Krembil Research Institute (Photo by Tim Fraser).