A new study released by the US Centers for Disease Control is under fire over both the study’s methodology and its presentation, just two days after its publication.
On Friday, the CDC published the study in its own journal, the Morbidity and Mortality Weekly Report (MMWR).
For the study, CDC researchers telephoned hundreds of California residents who were tested for SARS-CoV-2 from last February till the beginning of December.
A total of 652 people who tested positive for SARS-CoV-2 were surveyed, compared to 1,176 people who tested negative.
Researchers matched study participants in the negative and positive cohorts based on age groups, gender, and region within California. Participants were called two days after they received the results of their tests.
Participants interviewed over the phone were asked to self-report on how often they wore face masks while in indoor public venues, and which type they generally wore. Researchers then compared the rates at which respondents with negative tests wore masks, broke down into three categories, versus the rates at which those with positive tests said they had worn masks.
The study, which screened for people who had previously tested positive for the coronavirus, found that use of N95 (or KN95) respirators while participants were visiting indoor public venues had a statistically significant impact in reducing their odds of testing positive for SARS-CoV-2.
While surgical masks had a smaller impact, the study also found it to be statistically significant.
The impact of cloth masks, however, was not statistically significant, the report said.
The CDC quickly promoted the study, releasing an infographic showing the raw rate reduction data, while neglecting to note that cloth masks failed to cross the design threshold of the study for statistical significance.
The infographic quickly drew criticism, with critics pointing out the CDC report itself found cloth masks failed to show a statistically significant risk reduction for infection.
Some also lambasted the methodology of the study, including its reliance on self-reporting, the low participation rates, and confounders in the data.
Dr. Vinayak Prasad, a San Francisco-based hematologist-oncologist, health researcher, and associate professor at the University of California at San Francisco, excoriated the CDC report, calling it “entirely, irredeemably flawed.”
“Its flaws are so evident that it should not have been published nor promoted.”
In a critique of the study, Prasad noted that just 13.4% of people who tested positive whom researchers had called actually answered the phone and participated in the study, with just 8.9% of people in the negative test group answering the phone.
“When you get a response rate this low you wonder if you are inserting biases you cannot even imagine. Who are the sick COVID people who answer the phone. Are they the least sick ones? The sick ones don’t take calls. Who are the people who answer the phone who test negative? The most anxious ones? The most gullible?”
Prasad also noted that respondents who said they had not spent time indoor settings were removed from the study, as were people who had a known COVID-positive contact.
More significantly, respondents who tested positive tended to undergo testing for different reasons than respondents in the negative cohort, with 78% of positive respondents getting tested because they had symptoms, compared to just 17% of the negative cohort, which had a larger percentage of people who tested just to check if they were infected, because they are required to undergo routine screening, because they are planning to travel, or because they are undergoing a medical procedure.
“Put more technically, the root virtue of test negative strategy (comparability) is violated and unmeasured confounding is injected. Paper cannot be salvaged.”
Prasad further pointed out that self-reporting behavior after results constituted a “huge bias”.
He also noted that randomized controlled studies either found no significant risk reduction, or far smaller risk reductions, and pointed out that a 2020 CDC study did not suggest any strong connection between masking and risk reduction for SARS-CoV-2 infections.